TiDB 新功能 S3 存算分离：如何节省 90% 成本

本地搭建s3节点做分离计算。

minio server /tmp/minio_data --address :9222

sudo mkdir -p /tmp/minio_data/tiflash-s3

sudo tiup playground \

--db.config /tmp/tiflash-config/tidb.toml \

--tiflash.config /tmp/tiflash-config/tiflash-wn.toml

sudo tiup tiflash:v7.2.0 server \

--config-file /tmp/tiflash-config/tiflash-rn.toml

sudo tiup bench tpcc -H 127.0.0.1 -P 4000 -D tpcc --warehouses 10 --threads 20 prepare

mysql -h 127.0.0.1 -P 4000 -u root

ALTER DATABASE tpcc SET TIFLASH REPLICA 1;

SELECT * FROM information_schema.tiflash_replica WHERE TABLE_SCHEMA = tpcc;

analyze table customer; analyze table district; analyze table history; analyze table item; analyze table new_order; analyze table order_line; analyze table orders; analyze table stock; analyze table warehouse; analyze table nation; analyze table region; analyze table supplier;

sudo tiup bench tpcc -H 127.0.0.1 -P 4000 -D tpcc --warehouses 10 --threads 100 --time 10m run

[Summary] DELIVERY - Takes(s): 63.6, Count: 748, TPM: 706.0, Sum(ms): 429992.4, Avg(ms): 574.3, 50th(ms): 486.5, 90th(ms): 973.1, 95th(ms): 1275.1, 99th(ms): 1946.2, 99.9th(ms): 2415.9, Max(ms): 2415.9

[Summary] DELIVERY_ERR - Takes(s): 63.6, Count: 9, TPM: 8.5, Sum(ms): 2337.3, Avg(ms): 259.2, 50th(ms): 285.2, 90th(ms): 419.4, 95th(ms): 419.4, 99th(ms): 419.4, 99.9th(ms): 419.4, Max(ms): 419.4

[Summary] NEW_ORDER - Takes(s): 64.2, Count: 8304, TPM: 7765.4, Sum(ms): 2278813.2, Avg(ms): 274.6, 50th(ms): 167.8, 90th(ms): 637.5, 95th(ms): 872.4, 99th(ms): 1409.3, 99.9th(ms): 1811.9, Max(ms): 2080.4

[Summary] NEW_ORDER_ERR - Takes(s): 64.2, Count: 27, TPM: 25.2, Sum(ms): 9199.3, Avg(ms): 341.7, 50th(ms): 184.5, 90th(ms): 872.4, 95th(ms): 1140.9, 99th(ms): 1208.0, 99.9th(ms): 1208.0, Max(ms): 1208.0

[Summary] ORDER_STATUS - Takes(s): 64.3, Count: 746, TPM: 695.8, Sum(ms): 38576.1, Avg(ms): 51.7, 50th(ms): 35.7, 90th(ms): 104.9, 95th(ms): 151.0, 99th(ms): 335.5, 99.9th(ms): 503.3, Max(ms): 604.0

[Summary] ORDER_STATUS_ERR - Takes(s): 64.3, Count: 2, TPM: 1.9, Sum(ms): 50.7, Avg(ms): 25.6, 50th(ms): 4.2, 90th(ms): 48.2, 95th(ms): 48.2, 99th(ms): 48.2, 99.9th(ms): 48.2, Max(ms): 48.2

[Summary] PAYMENT - Takes(s): 64.4, Count: 8057, TPM: 7507.6, Sum(ms): 3561426.5, Avg(ms): 442.1, 50th(ms): 352.3, 90th(ms): 906.0, 95th(ms): 1208.0, 99th(ms): 1610.6, 99.9th(ms): 1946.2, Max(ms): 2080.4

[Summary] PAYMENT_ERR - Takes(s): 64.4, Count: 56, TPM: 52.2, Sum(ms): 25858.9, Avg(ms): 462.9, 50th(ms): 318.8, 90th(ms): 1073.7, 95th(ms): 1140.9, 99th(ms): 1208.0, 99.9th(ms): 1476.4, Max(ms): 1476.4

[Summary] STOCK_LEVEL - Takes(s): 64.1, Count: 747, TPM: 699.0, Sum(ms): 69036.7, Avg(ms): 92.4, 50th(ms): 37.7, 90th(ms): 302.0, 95th(ms): 436.2, 99th(ms): 671.1, 99.9th(ms): 838.9, Max(ms): 906.0

[Summary] STOCK_LEVEL_ERR - Takes(s): 64.1, Count: 1, TPM: 0.9, Sum(ms): 15.2, Avg(ms): 14.9, 50th(ms): 15.2, 90th(ms): 15.2, 95th(ms): 15.2, 99th(ms): 15.2, 99.9th(ms): 15.2, Max(ms): 15.2

tpmC: 7763.0, tpmTotal: 17368.3, efficiency: 6036.5%

本地自建一体式tiflash

sudo tiup playground

sudo tiup bench tpcc -H 127.0.0.1 -P 4000 -D tpcc --warehouses 10 --threads 20 prepare

mysql -h 127.0.0.1 -P 4000 -u root

ALTER DATABASE tpcc SET TIFLASH REPLICA 1;

SELECT * FROM information_schema.tiflash_replica WHERE TABLE_SCHEMA = tpcc;

analyze table customer; analyze table district; analyze table history; analyze table item; analyze table new_order; analyze table order_line; analyze table orders; analyze table stock; analyze table warehouse; analyze table nation; analyze table region; analyze table supplier;

sudo tiup bench tpcc -H 127.0.0.1 -P 4000 -D tpcc --warehouses 10 --threads 100 --time 10m run

[Summary] DELIVERY - Takes(s): 49.4, Count: 634, TPM: 769.8, Sum(ms): 321734.9, Avg(ms): 508.2, 50th(ms): 419.4, 90th(ms): 872.4, 95th(ms): 1140.9, 99th(ms): 1543.5, 99.9th(ms): 1811.9, Max(ms): 1946.2

[Summary] DELIVERY_ERR - Takes(s): 49.4, Count: 10, TPM: 12.1, Sum(ms): 5656.0, Avg(ms): 568.1, 50th(ms): 469.8, 90th(ms): 872.4, 95th(ms): 1476.4, 99th(ms): 1476.4, 99.9th(ms): 1476.4, Max(ms): 1476.4

[Summary] NEW_ORDER - Takes(s): 50.0, Count: 7039, TPM: 8439.8, Sum(ms): 1830589.6, Avg(ms): 260.2, 50th(ms): 151.0, 90th(ms): 637.5, 95th(ms): 872.4, 99th(ms): 1409.3, 99.9th(ms): 1946.2, Max(ms): 2281.7

[Summary] NEW_ORDER_ERR - Takes(s): 50.0, Count: 37, TPM: 44.4, Sum(ms): 13200.8, Avg(ms): 356.3, 50th(ms): 176.2, 90th(ms): 738.2, 95th(ms): 939.5, 99th(ms): 1140.9, 99.9th(ms): 1140.9, Max(ms): 1140.9

[Summary] ORDER_STATUS - Takes(s): 50.0, Count: 590, TPM: 708.6, Sum(ms): 36030.2, Avg(ms): 61.0, 50th(ms): 35.7, 90th(ms): 151.0, 95th(ms): 201.3, 99th(ms): 302.0, 99.9th(ms): 402.7, Max(ms): 436.2

[Summary] ORDER_STATUS_ERR - Takes(s): 50.0, Count: 1, TPM: 1.2, Sum(ms): 144.9, Avg(ms): 146.8, 50th(ms): 151.0, 90th(ms): 151.0, 95th(ms): 151.0, 99th(ms): 151.0, 99.9th(ms): 151.0, Max(ms): 151.0

[Summary] PAYMENT - Takes(s): 50.1, Count: 6802, TPM: 8150.6, Sum(ms): 2724614.0, Avg(ms): 400.7, 50th(ms): 318.8, 90th(ms): 838.9, 95th(ms): 1140.9, 99th(ms): 1610.6, 99.9th(ms): 2147.5, Max(ms): 2281.7

[Summary] PAYMENT_ERR - Takes(s): 50.1, Count: 41, TPM: 49.1, Sum(ms): 16560.9, Avg(ms): 403.2, 50th(ms): 402.7, 90th(ms): 1006.6, 95th(ms): 1073.7, 99th(ms): 1342.2, 99.9th(ms): 1342.2, Max(ms): 1342.2

[Summary] STOCK_LEVEL - Takes(s): 49.9, Count: 610, TPM: 732.7, Sum(ms): 40575.3, Avg(ms): 66.5, 50th(ms): 28.3, 90th(ms): 117.4, 95th(ms): 352.3, 99th(ms): 671.1, 99.9th(ms): 872.4, Max(ms): 1040.2

[Summary] STOCK_LEVEL_ERR - Takes(s): 49.9, Count: 1, TPM: 1.2, Sum(ms): 25.4, Avg(ms): 25.7, 50th(ms): 26.2, 90th(ms): 26.2, 95th(ms): 26.2, 99th(ms): 26.2, 99.9th(ms): 26.2, Max(ms): 26.2

tpmC: 8439.7, tpmTotal: 18801.3, efficiency: 6562.7%

s3 在容量大幅度减少的情况下获得了全量盘92%的性能。这些都是钱呀

这是在计算资源都在一起的情况下 其实真实业务是tilash的性能会更好一点。因为写节点的机器是单独建立的。而计算是读取写节点的数据加上缓存节点的数据。总的cpu核心数会比一体式ap计算更多。

到时候我在测试一下aws上的真实性能。