hello,
I have executed the command “juju run-action --wait mysql-innodb-cluster/1 reboot-cluster-from-complete-outage” but mysql-innodb-cluster/1 status still blocked
If I check /var/log/juju/unit-mysql-innodb-cluster-1.log like as bellow :
2022-07-01 08:14:05 INFO unit.mysql-innodb-cluster/1.juju-log server.go:319 Invoking reactive handler: hooks/relations/mysql-router/provides.py:47:joined:db-router 2022-07-01 08:14:05 INFO unit.mysql-innodb-cluster/1.juju-log server.go:319 DEPRECATION WARNING: Function _ows_check_services_running is being removed on/around 2022-05 : use ows_check_services_running() instead 2022-07-01 08:14:32 ERROR unit.mysql-innodb-cluster/1.juju-log server.go:319 Cluster is unavailable: Cannot set LC_ALL to locale en_US.UTF-8: No such file or directory Traceback (most recent call last): File “”, line 2, in RuntimeError: Dba.get_cluster: Group replication does not seem to be active in instance ‘192.168.80.32:3306’
2022-07-01 08:14:32 INFO unit.mysql-innodb-cluster/1.juju-log server.go:319 coordinator.DelayedActionCoordinator Publishing state 2022-07-01 08:14:32 INFO juju.worker.uniter.operation runhook.go:146 ran “update-status” hook (via explicit, bespoke hook script)
If I check /var/log/mysql/error.log like as bellow :
root@juju-212891-1-lxd-0:/var/log/mysql# tail -f error.log 2022-07-01T08:00:37.380694Z 1575 [Warning] [MY-010584] [Repl] Slave SQL for channel ‘group_replication_recovery’: … The slave coordinator and worker threads are stopped, possibly leaving data in inconsistent state. A restart should restore consistency automatically, although using non-transactional storage for data or info tables or DDL queries could lead to problems. In such cases you have to examine your data (see documentation for details). Error_code: MY-001756 2022-07-01T08:00:37.424327Z 1509 [System] [MY-010597] [Repl] ‘CHANGE MASTER TO FOR CHANNEL ‘group_replication_recovery’ executed’. Previous state master_host=‘192.168.80.33’, master_port= 3306, master_log_file=’’, master_log_pos= 4, master_bind=’’. New state master_host=‘192.168.80.30’, master_port= 3306, master_log_file=’’, master_log_pos= 4, master_bind=’’. 2022-07-01T08:00:37.441942Z 1580 [Warning] [MY-010897] [Repl] Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the ‘START SLAVE Syntax’ in the MySQL Manual for more information. 2022-07-01T08:00:37.451013Z 1580 [System] [MY-010562] [Repl] Slave I/O thread for channel ‘group_replication_recovery’: connected to master ‘mysql_innodb_cluster_1001@192.168.80.30:3306’,replication started in log ‘FIRST’ at position 4 2022-07-01T08:00:37.478418Z 1582 [ERROR] [MY-010584] [Repl] Slave SQL for channel ‘group_replication_recovery’: Worker 1 failed executing transaction ‘3fd8b69d-e584-11ec-b4f4-00163e488163:4176114’ at master log binlog.000047, end_log_pos 1065278; Could not execute Update_rows event on table horizon.django_session; Can’t find record in ‘django_session’, Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event’s master log FIRST, end_log_pos 1065278, Error_code: MY-001032 2022-07-01T08:00:37.478873Z 1581 [ERROR] [MY-010586] [Repl] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with “SLAVE START”. We stopped at log ‘binlog.000047’ position 1044671 2022-07-01T08:00:37.480666Z 1509 [ERROR] [MY-011574] [Repl] Plugin group_replication reported: ‘Maximum number of retries when trying to connect to a donor reached. Aborting group replication incremental recovery.’ 2022-07-01T08:00:37.480728Z 1509 [ERROR] [MY-011620] [Repl] Plugin group_replication reported: ‘Fatal error during the incremental recovery process of Group Replication. The server will leave the group.’ 2022-07-01T08:00:37.480829Z 1509 [ERROR] [MY-011712] [Repl] Plugin group_replication reported: ‘The server was automatically set into read only mode after an error was detected.’ 2022-07-01T08:00:40.573120Z 0 [System] [MY-011504] [Repl] Plugin group_replication reported: ‘Group membership changed: This member has left the group.’
please help to fix this problem.
Thanks.