[大数据] 迁移 MySQL 数据到 OceanBase 集群

开发: C++知识库 Java知识库 JavaScript Python PHP知识库人工智能区块链大数据移动开发嵌入式开发工具数据结构与算法开发测试游戏开发网络协议系统运维
教程: HTML教程 CSS教程 JavaScript教程 Go语言教程 JQuery教程 VUE教程 VUE3教程 Bootstrap教程 SQL数据库教程 C语言教程 C++教程 Java教程 Python教程 Python3教程 C#教程
数码: 电脑笔记本显卡显示器固态硬盘硬盘耳机手机 iphone vivo oppo 小米华为单反装机图拉丁

-> 大数据 -> 迁移 MySQL 数据到 OceanBase 集群 -> 正文阅读

[大数据]迁移 MySQL 数据到 OceanBase 集群

一、准备数据

1.下载编译mysql tpcc测试程序及脚本，并进行编译。

[admin@10 ~]$ git clone https://github.com/Percona-Lab/tpcc-mysql.git
Cloning into 'tpcc-mysql'...
remote: Enumerating objects: 392, done.
remote: Total 392 (delta 0), reused 0 (delta 0), pack-reused 392
Receiving objects: 100% (392/392), 202.81 KiB | 910.00 KiB/s, done.
Resolving deltas: 100% (216/216), done.
[admin@10 ~]$ ls
obdeploy  obproxy-3.1.0  oceanbase  tpcc-mysql
[admin@10 ~]$ cd tpcc-mysql/
[admin@10 tpcc-mysql]$ ls
add_fkey_idx.sql  count.sql  create_table.sql  Dockerfile  drop_cons.sql  load_multi_schema.sh  load.sh  README.md  schema2  scripts  src
[admin@10 tpcc-mysql]$ cd src/
[admin@10 src]$ ls
delivery.c  load.c  Makefile  ordstat.c     payment.c  rthist.h         sb_percentile.h  sequence.h  spt_proc.c  support.c  trans_if.h
driver.c    main.c  neword.c  parse_port.h  rthist.c   sb_percentile.c  sequence.c       slev.c      spt_proc.h  tpc.h
[admin@10 src]$ make
cc -w -O3 -g -I. `mysql_config --include`  -c load.c
cc -w -O3 -g -I. `mysql_config --include`  -c support.c
cc load.o support.o `mysql_config --libs_r` -lrt -o ../tpcc_load
cc -w -O3 -g -I. `mysql_config --include`  -c main.c
cc -w -O3 -g -I. `mysql_config --include`  -c spt_proc.c
cc -w -O3 -g -I. `mysql_config --include`  -c driver.c
cc -w -O3 -g -I. `mysql_config --include`  -c sequence.c
cc -w -O3 -g -I. `mysql_config --include`  -c rthist.c
cc -w -O3 -g -I. `mysql_config --include`  -c sb_percentile.c
cc -w -O3 -g -I. `mysql_config --include`  -c neword.c
cc -w -O3 -g -I. `mysql_config --include`  -c payment.c
cc -w -O3 -g -I. `mysql_config --include`  -c ordstat.c
cc -w -O3 -g -I. `mysql_config --include`  -c delivery.c
cc -w -O3 -g -I. `mysql_config --include`  -c slev.c
cc main.o spt_proc.o driver.o support.o sequence.o rthist.o sb_percentile.o neword.o payment.o ordstat.o delivery.o slev.o `mysql_config --libs_r` -lrt -o ../tpcc_start

2.创建数据库，并导入tpcc建表脚本

[admin@10 ~]$ mysql -uroot
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 8
Server version: 10.3.9-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
+--------------------+
3 rows in set (0.000 sec)
MariaDB [(none)]> create database tpcc_test;
Query OK, 1 row affected (0.000 sec)
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| tpcc_test          |
+--------------------+
4 rows in set (0.000 sec)
MariaDB [(none)]> use tpcc_test;
Database changed
MariaDB [tpcc_test]> source /home/admin/tpcc-mysql/create_table.sql;
Query OK, 0 rows affected (0.000 sec)
Query OK, 0 rows affected (0.000 sec)
Query OK, 0 rows affected, 1 warning (0.001 sec)
Query OK, 0 rows affected (0.016 sec)
Query OK, 0 rows affected, 1 warning (0.000 sec)
Query OK, 0 rows affected (0.008 sec)
Query OK, 0 rows affected, 1 warning (0.000 sec)
Query OK, 0 rows affected (0.009 sec)
Query OK, 0 rows affected, 1 warning (0.000 sec)
Query OK, 0 rows affected (0.008 sec)
Query OK, 0 rows affected, 1 warning (0.001 sec)
Query OK, 0 rows affected (0.009 sec)
Query OK, 0 rows affected, 1 warning (0.000 sec)
Query OK, 0 rows affected (0.013 sec)
Query OK, 0 rows affected, 1 warning (0.002 sec)
Query OK, 0 rows affected (0.011 sec)
Query OK, 0 rows affected, 1 warning (0.000 sec)
Query OK, 0 rows affected (0.008 sec)
Query OK, 0 rows affected, 1 warning (0.000 sec)
Query OK, 0 rows affected (0.007 sec)
Query OK, 0 rows affected (0.000 sec)
Query OK, 0 rows affected (0.000 sec)
MariaDB [tpcc_test]>

3.导入测试数据

./tpcc_load -h127.0.0.1 -P3306 -d tpcc_test -uroot -w 2 -p123456

二、使用 mysqldump 将 mysql的表结构和数据同步到 OceanBase 的MySQL 租户中
1.导出mysql数据

# 导出表结构
[admin@10 tpcc-mysql]$ mysqldump -h127.0.0.1 -P2206 -p123456 -uroot -d tpcc_test --compact > tpcc_ddl.sql
# 导出数据
[admin@10 tpcc-mysql]$ mysqldump -h127.0.0.1 -P3306 -p123456 -uroot -t tpcc_test --compact > tpcc_data.sql

2.导入OceanBase

-- 导入表结构
MySQL [(none)]> use obtest
Database changed
MySQL [obtest]> source /home/admin/tpcc-mysql/tpcc_ddl.sql
Query OK, 1755 rows affected (0.909 sec)
Records: 1755  Duplicates: 0  Warnings: 0
.............
-- 导入数据
MySQL [obtest]> source /home/admin/tpcc-mysql/tpcc_data.sql
Query OK, 3162 rows affected (1.406 sec)
Records: 3162  Duplicates: 0  Warnings: 0
Query OK, 3162 rows affected (1.621 sec)
Records: 3162  Duplicates: 0  Warnings: 0
...............

3.验证

MySQL [(none)]> use obtest;
Database changed
MySQL [obtest]> show tables;
+------------------+
| Tables_in_obtest |
+------------------+
| customer         |
| district         |
| history          |
| item             |
| new_orders       |
| order_line       |
| orders           |
| stock            |
| t1               |
| t2               |
| warehouse        |
+------------------+
11 rows in set (0.003 sec)

三、使用 datax 配置至少一个表的 OceanBase 到 CSV 以及 CSV 到 OceanBase 的离线同步。

1.mysql to csv

[admin@10 datax]$ bin/datax.py job/my2csv.json
DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.
2022-02-01 15:04:36.306 [main] INFO  VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2022-02-01 15:04:36.325 [main] INFO  Engine - the machine info  =>
        osInfo: openEuler Community 1.8 25.272-b10
        jvmInfo:        Linux amd64 4.19.90-24.4.v2101.ky10.x86_64
        cpu num:        6
        totalPhysicalMemory:    -0.00G
        freePhysicalMemory:     -0.00G
        maxFileDescriptorCount: -1
        currentOpenFileDescriptorCount: -1
        GC Names        [PS MarkSweep, PS Scavenge]
        MEMORY_NAME                    | allocation_size                | init_size
        PS Eden Space                  | 256.00MB                       | 256.00MB
        Code Cache                     | 240.00MB                       | 2.44MB
        Compressed Class Space         | 1,024.00MB                     | 0.00MB
        PS Survivor Space              | 42.50MB                        | 42.50MB
        PS Old Gen                     | 683.00MB                       | 683.00MB
        Metaspace                      | -0.00MB                        | 0.00MB
2022-02-01 15:04:36.360 [main] INFO  Engine -
{
        "content":[
                {
                        "reader":{
                                "name":"mysqlreader",
                                "parameter":{
                                        "column":[
                                                "*"
                                        ],
                                        "connection":[
                                                {
                                                        "jdbcUrl":[
                                                                "jdbc:mysql://127.0.0.1:3306/tpcc_test?useUnicode=true&characterEncoding=utf8&useSSL=false"
                                                        ],
                                                        "table":[
                                                                "customer"
                                                        ]
                                                }
                                        ],
                                        "password":"******",
                                        "username":"root"
                                }
                        },
                        "writer":{
                                "name":"txtfilewriter",
                                "parameter":{
                                        "dateFormat":"yyyy-MM-dd hh:mm:ss",
                                        "encoding":"UTF-8",
                                        "fieldDelimiter":",",
                                        "fileFormat":"csv",
                                        "fileName":"customer",
                                        "nullFormat":"\\N",
                                        "path":"/home/admin/csv",
                                        "writeMode":"truncate"
                                }
                        }
                }
        ],
        "setting":{
                "errorLimit":{
                        "percentage":0.1,
                        "record":0
                },
                "speed":{
                        "channel":4
                }
        }
}
2022-02-01 15:04:36.401 [main] WARN  Engine - prioriy set to 0, because NumberFormatException, the value is: null
2022-02-01 15:04:36.403 [main] INFO  PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2022-02-01 15:04:36.404 [main] INFO  JobContainer - DataX jobContainer starts job.
2022-02-01 15:04:36.408 [main] INFO  JobContainer - Set jobId = 0
2022-02-01 15:04:37.155 [job-0] INFO  OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://127.0.0.1:3306/tpcc_test?useUnicode=true&characterEncoding=utf8&useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2022-02-01 15:04:37.157 [job-0] WARN  OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2022-02-01 15:04:37.402 [job-0] INFO  JobContainer - jobContainer starts to do prepare ...
2022-02-01 15:04:37.403 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2022-02-01 15:04:37.404 [job-0] INFO  JobContainer - DataX Writer.Job [txtfilewriter] do prepare work .
2022-02-01 15:04:37.404 [job-0] INFO  TxtFileWriter$Job - 由于您配置了writeMode truncate, 开始清理 [/home/admin/csv] 下面以 [customer] 开头的内容
2022-02-01 15:04:37.408 [job-0] INFO  JobContainer - jobContainer starts to do split ...
2022-02-01 15:04:37.409 [job-0] INFO  JobContainer - Job set Channel-Number to 4 channels.
2022-02-01 15:04:37.413 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] splits to [1] tasks.
2022-02-01 15:04:37.414 [job-0] INFO  TxtFileWriter$Job - begin do split...
2022-02-01 15:04:37.427 [job-0] INFO  TxtFileWriter$Job - splited write file name:[customer__537abe0a_125d_423e_82ea_68739482a9ba]
2022-02-01 15:04:37.428 [job-0] INFO  TxtFileWriter$Job - end do split.
2022-02-01 15:04:37.428 [job-0] INFO  JobContainer - DataX Writer.Job [txtfilewriter] splits to [1] tasks.
2022-02-01 15:04:37.450 [job-0] INFO  JobContainer - jobContainer starts to do schedule ...
2022-02-01 15:04:37.454 [job-0] INFO  JobContainer - Scheduler starts [1] taskGroups.
2022-02-01 15:04:37.456 [job-0] INFO  JobContainer - Running by standalone Mode.
2022-02-01 15:04:37.469 [taskGroup-0] INFO  TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks.
2022-02-01 15:04:37.474 [taskGroup-0] INFO  Channel - Channel set byte_speed_limit to -1, No bps activated.
2022-02-01 15:04:37.475 [taskGroup-0] INFO  Channel - Channel set record_speed_limit to -1, No tps activated.
2022-02-01 15:04:37.496 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2022-02-01 15:04:37.496 [0-0-0-writer] INFO  TxtFileWriter$Task - begin do write...
2022-02-01 15:04:37.497 [0-0-0-writer] INFO  TxtFileWriter$Task - write to file : [/home/admin/csv/customer__537abe0a_125d_423e_82ea_68739482a9ba]
2022-02-01 15:04:37.504 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from customer
] jdbcUrl:[jdbc:mysql://127.0.0.1:3306/tpcc_test?useUnicode=true&characterEncoding=utf8&useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-02-01 15:04:39.720 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from customer
] jdbcUrl:[jdbc:mysql://127.0.0.1:3306/tpcc_test?useUnicode=true&characterEncoding=utf8&useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-02-01 15:04:39.725 [0-0-0-writer] INFO  TxtFileWriter$Task - end do write
2022-02-01 15:04:39.803 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[2309]ms
2022-02-01 15:04:39.805 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] completed it's tasks.
2022-02-01 15:04:47.485 [job-0] INFO  StandAloneJobContainerCommunicator - Total 60000 records, 32139281 bytes | Speed 3.07MB/s, 6000 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.534s |  All Task WaitReaderTime 0.153s | Percentage 100.00%
2022-02-01 15:04:47.485 [job-0] INFO  AbstractScheduler - Scheduler accomplished all tasks.
2022-02-01 15:04:47.486 [job-0] INFO  JobContainer - DataX Writer.Job [txtfilewriter] do post work.
2022-02-01 15:04:47.486 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do post work.
2022-02-01 15:04:47.486 [job-0] INFO  JobContainer - DataX jobId [0] completed successfully.
2022-02-01 15:04:47.487 [job-0] INFO  HookInvoker - No hook invoked, because base dir not exists or is a file: /home/admin/datax/hook
2022-02-01 15:04:47.488 [job-0] INFO  JobContainer -
         [total cpu info] =>
                averageCpu                     | maxDeltaCpu                    | minDeltaCpu
                -1.00%                         | -1.00%                         | -1.00%
         [total gc info] =>
                 NAME                 | totalGCCount       | maxDeltaGCCount    | minDeltaGCCount    | totalGCTime        | maxDeltaGCTime     | minDeltaGCTime
                 PS MarkSweep         | 0                  | 0                  | 0                  | 0.000s             | 0.000s             | 0.000s
                 PS Scavenge          | 1                  | 1                  | 1                  | 0.029s             | 0.029s             | 0.029s
2022-02-01 15:04:47.488 [job-0] INFO  JobContainer - PerfTrace not enable!
2022-02-01 15:04:47.488 [job-0] INFO  StandAloneJobContainerCommunicator - Total 60000 records, 32139281 bytes | Speed 3.07MB/s, 6000 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.534s |  All Task WaitReaderTime 0.153s | Percentage 100.00%
2022-02-01 15:04:47.490 [job-0] INFO  JobContainer -
任务启动时刻                    : 2022-02-01 15:04:36
任务结束时刻                    : 2022-02-01 15:04:47
任务总计耗时                    :                 11s
任务平均流量                    :            3.07MB/s
记录写入速度                    :           6000rec/s
读出记录总数                    :               60000
读写失败总数                    :                   0

2.csv to OceanBase

[admin@10 datax]$ bin/datax.py job/csv2ob.json
DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.
2022-02-01 15:34:53.231 [main] INFO  VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2022-02-01 15:34:53.277 [main] INFO  Engine - the machine info  =>
        osInfo: openEuler Community 1.8 25.272-b10
        jvmInfo:        Linux amd64 4.19.90-24.4.v2101.ky10.x86_64
        cpu num:        6
        totalPhysicalMemory:    -0.00G
        freePhysicalMemory:     -0.00G
        maxFileDescriptorCount: -1
        currentOpenFileDescriptorCount: -1
        GC Names        [PS MarkSweep, PS Scavenge]
        MEMORY_NAME                    | allocation_size                | init_size
        PS Eden Space                  | 256.00MB                       | 256.00MB
        Code Cache                     | 240.00MB                       | 2.44MB
        Compressed Class Space         | 1,024.00MB                     | 0.00MB
        PS Survivor Space              | 42.50MB                        | 42.50MB
        PS Old Gen                     | 683.00MB                       | 683.00MB
        Metaspace                      | -0.00MB                        | 0.00MB
2022-02-01 15:34:53.337 [main] INFO  Engine -
{
        "content":[
                {
                        "reader":{
                                "name":"txtfilereader",
                                "parameter":{
                                        "column":[
                                                "*"
                                        ],
                                        "dateFormat":"yyyy-MM-dd hh:mm:ss",
                                        "encoding":"UTF-8",
                                        "fieldDelimiter":",",
                                        "fileName":"customer__537abe0a_125d_423e_82ea_68739482a9ba",
                                        "nullFormat":"\\N",
                                        "path":[
                                                "/home/admin/csv"
                                        ]
                                }
                        },
                        "writer":{
                                "name":"oceanbasev10writer",
                                "parameter":{
                                        "batchSize":1000,
                                        "column":[
                                                "*"
                                        ],
                                        "connection":[
                                                {
                                                        "jdbcUrl":"||_dsc_ob10_dsc_||obdemo:sys||_dsc_ob10_dsc_||jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true",
                                                        "table":[
                                                                "customer"
                                                        ]
                                                }
                                        ],
                                        "memstoreThreshold":"0.9",
                                        "obWriteMode":"insert",
                                        "password":"******",
                                        "preSql":[
                                                "truncate table customer"
                                        ],
                                        "username":"root",
                                        "writerThreadCount":10
                                }
                        }
                }
        ],
        "setting":{
                "errorLimit":{
                        "percentage":0.1,
                        "record":0
                },
                "speed":{
                        "channel":4
                }
        }
}
2022-02-01 15:34:53.382 [main] WARN  Engine - prioriy set to 0, because NumberFormatException, the value is: null
2022-02-01 15:34:53.388 [main] INFO  PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2022-02-01 15:34:53.389 [main] INFO  JobContainer - DataX jobContainer starts job.
2022-02-01 15:34:53.396 [main] INFO  JobContainer - Set jobId = 0
2022-02-01 15:34:53.470 [job-0] INFO  DBUtil - this is ob1_0 jdbc url.
2022-02-01 15:34:53.470 [job-0] INFO  DBUtil - this is ob1_0 jdbc url. user=obdemo:sys:root :url=jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true
obproxy Druid LogFactory, userDefinedLogType=null, logInfo=public com.alipay.oceanbase.obproxy.util.log.CommonLoggerComponent(java.lang.String)
2022-02-01 15:34:54.454 [job-0] INFO  DbUtils - value for query [SHOW VARIABLES LIKE 'ob_compatibility_mode'] is [MYSQL]
2022-02-01 15:34:54.477 [job-0] INFO  DBUtil - this is ob1_0 jdbc url.
2022-02-01 15:34:54.478 [job-0] INFO  DBUtil - this is ob1_0 jdbc url. user=obdemo:sys:root :url=jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-02-01 15:34:54.519 [job-0] INFO  OriginalConfPretreatmentUtil - table:[customer] all columns:[
c_id,c_d_id,c_w_id,c_first,c_middle,c_last,c_street_1,c_street_2,c_city,c_state,c_zip,c_phone,c_since,c_credit,c_credit_lim,c_discount,c_balance,c_ytd_payment,c_payment_cnt,c_delivery_cnt,c_data
].
2022-02-01 15:34:54.520 [job-0] WARN  OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2022-02-01 15:34:54.523 [job-0] INFO  OriginalConfPretreatmentUtil - Write data [
INSERT INTO %s (c_id,c_d_id,c_w_id,c_first,c_middle,c_last,c_street_1,c_street_2,c_city,c_state,c_zip,c_phone,c_since,c_credit,c_credit_lim,c_discount,c_balance,c_ytd_payment,c_payment_cnt,c_delivery_cnt,c_data) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
], which jdbcUrl like:[||_dsc_ob10_dsc_||obdemo:sys||_dsc_ob10_dsc_||jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]
2022-02-01 15:34:54.524 [job-0] INFO  JobContainer - jobContainer starts to do prepare ...
2022-02-01 15:34:54.524 [job-0] INFO  JobContainer - DataX Reader.Job [txtfilereader] do prepare work .
2022-02-01 15:34:54.526 [job-0] INFO  TxtFileReader$Job - add file [/home/admin/csv/customer__537abe0a_125d_423e_82ea_68739482a9ba] as a candidate to be read.
2022-02-01 15:34:54.529 [job-0] INFO  TxtFileReader$Job - 您即将读取的文件数为: [1]
2022-02-01 15:34:54.530 [job-0] INFO  JobContainer - DataX Writer.Job [oceanbasev10writer] do prepare work .
2022-02-01 15:34:54.533 [job-0] INFO  DBUtil - this is ob1_0 jdbc url.
2022-02-01 15:34:54.533 [job-0] INFO  DBUtil - this is ob1_0 jdbc url. user=obdemo:sys:root :url=jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-02-01 15:34:54.569 [job-0] INFO  CommonRdbmsWriter$Job - Begin to execute preSqls:[truncate table customer]. context info:||_dsc_ob10_dsc_||obdemo:sys||_dsc_ob10_dsc_||jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2022-02-01 15:34:55.093 [job-0] INFO  DBUtil - this is ob1_0 jdbc url.
2022-02-01 15:34:55.095 [job-0] INFO  DBUtil - this is ob1_0 jdbc url. user=obdemo:sys:root :url=jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-02-01 15:34:55.134 [job-0] INFO  DbUtils - value for query [show variables like 'version'] is [3.1.0]
2022-02-01 15:34:55.135 [job-0] INFO  JobContainer - jobContainer starts to do split ...
2022-02-01 15:34:55.136 [job-0] INFO  JobContainer - Job set Channel-Number to 4 channels.
2022-02-01 15:34:55.138 [job-0] INFO  JobContainer - DataX Reader.Job [txtfilereader] splits to [1] tasks.
2022-02-01 15:34:55.142 [job-0] INFO  JobContainer - DataX Writer.Job [oceanbasev10writer] splits to [1] tasks.
2022-02-01 15:34:55.180 [job-0] INFO  JobContainer - jobContainer starts to do schedule ...
2022-02-01 15:34:55.203 [job-0] INFO  JobContainer - Scheduler starts [1] taskGroups.
2022-02-01 15:34:55.211 [job-0] INFO  JobContainer - Running by standalone Mode.
2022-02-01 15:34:55.238 [taskGroup-0] INFO  TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks.
2022-02-01 15:34:55.255 [taskGroup-0] INFO  Channel - Channel set byte_speed_limit to -1, No bps activated.
2022-02-01 15:34:55.255 [taskGroup-0] INFO  Channel - Channel set record_speed_limit to -1, No tps activated.
2022-02-01 15:34:55.270 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2022-02-01 15:34:55.271 [0-0-0-reader] INFO  TxtFileReader$Task - reading file : [/home/admin/csv/customer__537abe0a_125d_423e_82ea_68739482a9ba]
2022-02-01 15:34:55.274 [0-0-0-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-02-01 15:34:55.279 [0-0-0-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-02-01 15:34:55.292 [0-0-0-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-02-01 15:34:55.313 [0-0-0-writer] INFO  CommonRdbmsWriter$Task - write mode: insert
2022-02-01 15:34:55.313 [0-0-0-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO customer (c_id,c_d_id,c_w_id,c_first,c_middle,c_last,c_street_1,c_street_2,c_city,c_state,c_zip,c_phone,c_since,c_credit,c_credit_lim,c_discount,c_balance,c_ytd_payment,c_payment_cnt,c_delivery_cnt,c_data) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
2022-02-01 15:34:55.314 [0-0-0-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-02-01 15:34:55.314 [0-0-0-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obdemo:sys:root :url=jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-02-01 15:34:55.329 [0-0-0-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-02-01 15:34:55.331 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-02-01 15:34:55.351 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-02-01 15:34:55.411 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-02-01 15:34:55.422 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-02-01 15:34:55.434 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-02-01 15:34:55.477 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 6 insert task.
2022-02-01 15:34:55.487 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 7 insert task.
2022-02-01 15:34:55.499 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 8 insert task.
2022-02-01 15:34:55.513 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 9 insert task.
2022-02-01 15:34:55.528 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 10 insert task.
2022-02-01 15:34:55.543 [0-0-0-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-02-01 15:34:55.544 [0-0-0-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obdemo:sys:root :url=jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-02-01 15:34:55.562 [0-0-0-writer] INFO  ColumnMetaCache - fetch columnMeta of table customer success
2022-02-01 15:34:55.570 [0-0-0-reader] INFO  UnstructuredStorageReaderUtil - CsvReader使用默认值[{"captureRawRecord":true,"columnCount":0,"comment":"#","currentRecord":-1,"delimiter":",","escapeMode":1,"headerCount":0,"rawRecord":"","recordDelimiter":"\u0000","safetySwitch":false,"skipEmptyRecords":true,"textQualifier":"\"","trimWhitespace":true,"useComments":false,"useTextQualifier":true,"values":[]}],csvReaderConfig值为[null]
2022-02-01 15:34:55.950 [0-0-0-writer] INFO  CommonRdbmsWriter$Task - isMemstoreFull=false
2022-02-01 15:35:03.962 [0-0-0-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 60, finished = 55
2022-02-01 15:35:04.122 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[8853]ms
2022-02-01 15:35:04.123 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] completed it's tasks.
2022-02-01 15:35:05.254 [job-0] INFO  StandAloneJobContainerCommunicator - Total 60000 records, 32799281 bytes | Speed 3.13MB/s, 6000 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 3.871s |  All Task WaitReaderTime 2.437s | Percentage 100.00%
2022-02-01 15:35:05.255 [job-0] INFO  AbstractScheduler - Scheduler accomplished all tasks.
2022-02-01 15:35:05.255 [job-0] INFO  JobContainer - DataX Writer.Job [oceanbasev10writer] do post work.
2022-02-01 15:35:05.256 [job-0] INFO  JobContainer - DataX Reader.Job [txtfilereader] do post work.
2022-02-01 15:35:05.256 [job-0] INFO  JobContainer - DataX jobId [0] completed successfully.
2022-02-01 15:35:05.256 [job-0] INFO  HookInvoker - No hook invoked, because base dir not exists or is a file: /home/admin/datax/hook
2022-02-01 15:35:05.259 [job-0] INFO  JobContainer -
         [total cpu info] =>
                averageCpu                     | maxDeltaCpu                    | minDeltaCpu
                -1.00%                         | -1.00%                         | -1.00%
         [total gc info] =>
                 NAME                 | totalGCCount       | maxDeltaGCCount    | minDeltaGCCount    | totalGCTime        | maxDeltaGCTime     | minDeltaGCTime
                 PS MarkSweep         | 0                  | 0                  | 0                  | 0.000s             | 0.000s             | 0.000s
                 PS Scavenge          | 4                  | 4                  | 4                  | 0.427s             | 0.427s             | 0.427s
2022-02-01 15:35:05.259 [job-0] INFO  JobContainer - PerfTrace not enable!
2022-02-01 15:35:05.259 [job-0] INFO  StandAloneJobContainerCommunicator - Total 60000 records, 32799281 bytes | Speed 3.13MB/s, 6000 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 3.871s |  All Task WaitReaderTime 2.437s | Percentage 100.00%
2022-02-01 15:35:05.260 [job-0] INFO  JobContainer -
任务启动时刻                    : 2022-02-01 15:34:53
任务结束时刻                    : 2022-02-01 15:35:05
任务总计耗时                    :                 11s
任务平均流量                    :            3.13MB/s
记录写入速度                    :           6000rec/s
读出记录总数                    :               60000
读写失败总数                    :                   0

四、使用 datax 配置至少一个表的 MySQL 到 OceanBase 的 MySQL 租户的离线同步

[admin@10 datax]$ bin/datax.py job/my2ob.json
DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.
2022-02-01 15:39:51.040 [main] INFO  VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2022-02-01 15:39:51.058 [main] INFO  Engine - the machine info  =>
        osInfo: openEuler Community 1.8 25.272-b10
        jvmInfo:        Linux amd64 4.19.90-24.4.v2101.ky10.x86_64
        cpu num:        6
        totalPhysicalMemory:    -0.00G
        freePhysicalMemory:     -0.00G
        maxFileDescriptorCount: -1
        currentOpenFileDescriptorCount: -1
        GC Names        [PS MarkSweep, PS Scavenge]
        MEMORY_NAME                    | allocation_size                | init_size
        PS Eden Space                  | 256.00MB                       | 256.00MB
        Code Cache                     | 240.00MB                       | 2.44MB
        Compressed Class Space         | 1,024.00MB                     | 0.00MB
        PS Survivor Space              | 42.50MB                        | 42.50MB
        PS Old Gen                     | 683.00MB                       | 683.00MB
        Metaspace                      | -0.00MB                        | 0.00MB
2022-02-01 15:39:51.100 [main] INFO  Engine -
{
        "content":[
                {
                        "reader":{
                                "name":"mysqlreader",
                                "parameter":{
                                        "column":[
                                                "*"
                                        ],
                                        "connection":[
                                                {
                                                        "jdbcUrl":[
                                                                "jdbc:mysql://127.0.0.1:3306/tpcc_test?useUnicode=true&characterEncoding=utf8"
                                                        ],
                                                        "table":[
                                                                "customer"
                                                        ]
                                                }
                                        ],
                                        "password":"******",
                                        "username":"root"
                                }
                        },
                        "writer":{
                                "name":"oceanbasev10writer",
                                "parameter":{
                                        "batchSize":1000,
                                        "column":[
                                                "*"
                                        ],
                                        "connection":[
                                                {
                                                        "jdbcUrl":"||_dsc_ob10_dsc_||obdemo:sys||_dsc_ob10_dsc_||jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true",
                                                        "table":[
                                                                "customer"
                                                        ]
                                                }
                                        ],
                                        "memstoreThreshold":"0.9",
                                        "obWriteMode":"insert",
                                        "password":"******",
                                        "preSql":[
                                                "truncate table customer"
                                        ],
                                        "username":"root",
                                        "writerThreadCount":10
                                }
                        }
                }
        ],
        "setting":{
                "speed":{
                        "channel":2
                }
        }
}
2022-02-01 15:39:51.133 [main] WARN  Engine - prioriy set to 0, because NumberFormatException, the value is: null
2022-02-01 15:39:51.138 [main] INFO  PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2022-02-01 15:39:51.139 [main] INFO  JobContainer - DataX jobContainer starts job.
2022-02-01 15:39:51.145 [main] INFO  JobContainer - Set jobId = 0
2022-02-01 15:39:51.768 [job-0] INFO  OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://127.0.0.1:3306/tpcc_test?useUnicode=true&characterEncoding=utf8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2022-02-01 15:39:51.772 [job-0] WARN  OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2022-02-01 15:39:51.801 [job-0] INFO  DBUtil - this is ob1_0 jdbc url.
2022-02-01 15:39:51.801 [job-0] INFO  DBUtil - this is ob1_0 jdbc url. user=obdemo:sys:root :url=jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true
2022-02-01 15:39:52.314 [job-0] INFO  DbUtils - value for query [SHOW VARIABLES LIKE 'ob_compatibility_mode'] is [MYSQL]
2022-02-01 15:39:52.338 [job-0] INFO  DBUtil - this is ob1_0 jdbc url.
2022-02-01 15:39:52.338 [job-0] INFO  DBUtil - this is ob1_0 jdbc url. user=obdemo:sys:root :url=jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-02-01 15:39:52.371 [job-0] INFO  OriginalConfPretreatmentUtil - table:[customer] all columns:[
c_id,c_d_id,c_w_id,c_first,c_middle,c_last,c_street_1,c_street_2,c_city,c_state,c_zip,c_phone,c_since,c_credit,c_credit_lim,c_discount,c_balance,c_ytd_payment,c_payment_cnt,c_delivery_cnt,c_data
].
2022-02-01 15:39:52.372 [job-0] WARN  OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2022-02-01 15:39:52.374 [job-0] INFO  OriginalConfPretreatmentUtil - Write data [
INSERT INTO %s (c_id,c_d_id,c_w_id,c_first,c_middle,c_last,c_street_1,c_street_2,c_city,c_state,c_zip,c_phone,c_since,c_credit,c_credit_lim,c_discount,c_balance,c_ytd_payment,c_payment_cnt,c_delivery_cnt,c_data) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
], which jdbcUrl like:[||_dsc_ob10_dsc_||obdemo:sys||_dsc_ob10_dsc_||jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]
2022-02-01 15:39:52.380 [job-0] INFO  JobContainer - jobContainer starts to do prepare ...
2022-02-01 15:39:52.381 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2022-02-01 15:39:52.381 [job-0] INFO  JobContainer - DataX Writer.Job [oceanbasev10writer] do prepare work .
2022-02-01 15:39:52.382 [job-0] INFO  DBUtil - this is ob1_0 jdbc url.
2022-02-01 15:39:52.383 [job-0] INFO  DBUtil - this is ob1_0 jdbc url. user=obdemo:sys:root :url=jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-02-01 15:39:52.403 [job-0] INFO  CommonRdbmsWriter$Job - Begin to execute preSqls:[truncate table customer]. context info:||_dsc_ob10_dsc_||obdemo:sys||_dsc_ob10_dsc_||jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2022-02-01 15:39:52.534 [job-0] INFO  DBUtil - this is ob1_0 jdbc url.
2022-02-01 15:39:52.535 [job-0] INFO  DBUtil - this is ob1_0 jdbc url. user=obdemo:sys:root :url=jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-02-01 15:39:52.565 [job-0] INFO  DbUtils - value for query [show variables like 'version'] is [3.1.0]
2022-02-01 15:39:52.566 [job-0] INFO  JobContainer - jobContainer starts to do split ...
2022-02-01 15:39:52.568 [job-0] INFO  JobContainer - Job set Channel-Number to 2 channels.
2022-02-01 15:39:52.584 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] splits to [1] tasks.
2022-02-01 15:39:52.586 [job-0] INFO  JobContainer - DataX Writer.Job [oceanbasev10writer] splits to [1] tasks.
2022-02-01 15:39:52.659 [job-0] INFO  JobContainer - jobContainer starts to do schedule ...
2022-02-01 15:39:52.672 [job-0] INFO  JobContainer - Scheduler starts [1] taskGroups.
2022-02-01 15:39:52.679 [job-0] INFO  JobContainer - Running by standalone Mode.
2022-02-01 15:39:52.695 [taskGroup-0] INFO  TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks.
2022-02-01 15:39:52.704 [taskGroup-0] INFO  Channel - Channel set byte_speed_limit to -1, No bps activated.
2022-02-01 15:39:52.705 [taskGroup-0] INFO  Channel - Channel set record_speed_limit to -1, No tps activated.
2022-02-01 15:39:52.721 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2022-02-01 15:39:52.725 [0-0-0-writer] INFO  OceanBaseV10Writer$Task - tableNumber:1,writerTask Class:com.alibaba.datax.plugin.writer.oceanbasev10writer.task.ConcurrentTableWriterTask
2022-02-01 15:39:52.728 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Begin to read record by Sql: [select * from customer
] jdbcUrl:[jdbc:mysql://127.0.0.1:3306/tpcc_test?useUnicode=true&characterEncoding=utf8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-02-01 15:39:52.732 [0-0-0-writer] INFO  ConcurrentTableWriterTask - configure url is unavailable, use obclient for connections.
2022-02-01 15:39:52.763 [0-0-0-writer] INFO  ConcurrentTableWriterTask - Disable partition calculation feature.
2022-02-01 15:39:52.784 [0-0-0-writer] INFO  CommonRdbmsWriter$Task - write mode: insert
2022-02-01 15:39:52.784 [0-0-0-writer] INFO  ConcurrentTableWriterTask - writeRecordSql :INSERT INTO customer (c_id,c_d_id,c_w_id,c_first,c_middle,c_last,c_street_1,c_street_2,c_city,c_state,c_zip,c_phone,c_since,c_credit,c_credit_lim,c_discount,c_balance,c_ytd_payment,c_payment_cnt,c_delivery_cnt,c_data) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
2022-02-01 15:39:52.789 [0-0-0-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-02-01 15:39:52.791 [0-0-0-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obdemo:sys:root :url=jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-02-01 15:39:52.812 [0-0-0-writer] ERROR ConcurrentTableWriterTask - partCalculator is null
2022-02-01 15:39:52.813 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 1 insert task.
2022-02-01 15:39:52.835 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 2 insert task.
2022-02-01 15:39:52.850 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 3 insert task.
2022-02-01 15:39:52.866 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 4 insert task.
2022-02-01 15:39:52.889 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 5 insert task.
2022-02-01 15:39:52.910 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 6 insert task.
2022-02-01 15:39:52.951 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 7 insert task.
2022-02-01 15:39:52.975 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 8 insert task.
2022-02-01 15:39:53.016 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 9 insert task.
2022-02-01 15:39:53.062 [0-0-0-writer] INFO  ConcurrentTableWriterTask - start 10 insert task.
2022-02-01 15:39:53.099 [0-0-0-writer] INFO  DBUtil - this is ob1_0 jdbc url.
2022-02-01 15:39:53.100 [0-0-0-writer] INFO  DBUtil - this is ob1_0 jdbc url. user=obdemo:sys:root :url=jdbc:oceanbase://10.0.2.15:2883/obtest?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true
2022-02-01 15:39:53.134 [0-0-0-writer] INFO  ColumnMetaCache - fetch columnMeta of table customer success
2022-02-01 15:39:53.245 [0-0-0-writer] INFO  CommonRdbmsWriter$Task - isMemstoreFull=false
2022-02-01 15:39:59.680 [0-0-0-reader] INFO  CommonRdbmsReader$Task - Finished read record by Sql: [select * from customer
] jdbcUrl:[jdbc:mysql://127.0.0.1:3306/tpcc_test?useUnicode=true&characterEncoding=utf8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2022-02-01 15:39:59.698 [0-0-0-writer] INFO  ConcurrentTableWriterTask - ConcurrentTableWriter has put all task in queue, queueSize = 0,  total = 60, finished = 55
2022-02-01 15:40:00.004 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[7285]ms
2022-02-01 15:40:00.007 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] completed it's tasks.
2022-02-01 15:40:02.730 [job-0] INFO  StandAloneJobContainerCommunicator - Total 60000 records, 32139281 bytes | Speed 3.07MB/s, 6000 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 1.789s |  All Task WaitReaderTime 2.990s | Percentage 100.00%
2022-02-01 15:40:02.731 [job-0] INFO  AbstractScheduler - Scheduler accomplished all tasks.
2022-02-01 15:40:02.733 [job-0] INFO  JobContainer - DataX Writer.Job [oceanbasev10writer] do post work.
2022-02-01 15:40:02.735 [job-0] INFO  JobContainer - DataX Reader.Job [mysqlreader] do post work.
2022-02-01 15:40:02.735 [job-0] INFO  JobContainer - DataX jobId [0] completed successfully.
2022-02-01 15:40:02.739 [job-0] INFO  HookInvoker - No hook invoked, because base dir not exists or is a file: /home/admin/datax/hook
2022-02-01 15:40:02.744 [job-0] INFO  JobContainer -
         [total cpu info] =>
                averageCpu                     | maxDeltaCpu                    | minDeltaCpu
                -1.00%                         | -1.00%                         | -1.00%
         [total gc info] =>
                 NAME                 | totalGCCount       | maxDeltaGCCount    | minDeltaGCCount    | totalGCTime        | maxDeltaGCTime     | minDeltaGCTime
                 PS MarkSweep         | 0                  | 0                  | 0                  | 0.000s             | 0.000s             | 0.000s
                 PS Scavenge          | 4                  | 4                  | 4                  | 0.360s             | 0.360s             | 0.360s
2022-02-01 15:40:02.746 [job-0] INFO  JobContainer - PerfTrace not enable!
2022-02-01 15:40:02.748 [job-0] INFO  StandAloneJobContainerCommunicator - Total 60000 records, 32139281 bytes | Speed 3.07MB/s, 6000 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 1.789s |  All Task WaitReaderTime 2.990s | Percentage 100.00%
2022-02-01 15:40:02.751 [job-0] INFO  JobContainer -
任务启动时刻                    : 2022-02-01 15:39:51
任务结束时刻                    : 2022-02-01 15:40:02
任务总计耗时                    :                 11s
任务平均流量                    :            3.07MB/s
记录写入速度                    :           6000rec/s
读出记录总数                    :               60000
读写失败总数                    :                   0

五、问题

ERROR 1115 (42000) at line 3 in file: '/home/admin/tpcc-mysql/tpcc_ddl.sql': Unknown character set: 'latin1'
– 手动替换：将latin1替换为utf8
插件[mysqlreader,txtfilewriter]加载失败
– 删除plugin[reader,writer]目录下所有._开头的文件夹/文件rm -rf ._*

大数据最新文章

实现Kafka至少消费一次

亚马逊云科技：还在苦于ETL？Zero ETL的时代

初探MapReduce

【SpringBoot框架篇】32.基于注解+redis实现

Elasticsearch：如何减少 Elasticsearch 集

Go redis操作

Redis面试题

专题五 Redis高并发场景

基于GBase8s和Calcite的多数据源查询

Redis——底层数据结构原理

加:2022-02-03 01:16:27 更:2022-02-03 01:18:11

360图书馆购物三丰科技阅读网日历万年历 2024年11日历

-2024/11/24 14:06:07-

图片自动播放器
↓图片自动播放器↓

TxT小说阅读器
↓语音阅读,小说下载,古典文学↓

一键清除垃圾
↓轻轻一点,清除系统垃圾↓

图片批量下载器
↓批量下载图片,美女图库↓

网站联系: qq:121756557 email:121756557@qq.com IT数码