使用hivesql 批量count fdp库下表名开头为db_的表的数据量
时间: 2024-03-12 07:46:11 浏览: 139
可以使用如下的HiveSQL语句批量count fdp库下表名开头为db_的表的数据量:
```sql
USE fdp;
SET hive.exec.dynamic.partition.mode=nonstrict;
SET hive.exec.dynamic.partition=true;
SET hive.exec.max.dynamic.partitions=100000;
SET hive.exec.max.dynamic.partitions.pernode=100000;
SET hive.exec.max.created.files=100000;
SET mapred.reduce.tasks=8;
SET mapred.map.tasks=8;
SET hive.auto.convert.join=true;
SET hive.optimize.bucketmapjoin=true;
SET hive.optimize.bucketmapjoin.sortedmerge=true;
SET hive.optimize.bucketmapjoin.sortedmerge.bucketmapjoin=true;
SET hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat;
SET hive.map.aggr=true;
SET hive.merge.mapfiles=true;
SET hive.merge.mapredfiles=true;
SET hive.merge.size.per.task=256000000;
SET hive.merge.smallfiles.avgsize=16000000;
SET hive.merge.orcfile.stripe.level=true;
SET hive.merge.orcfile.stripe.level.threshold=0.5;
SET hive.merge.tezfiles=true;
SET hive.merge.smallfiles.avgsize=134217728;
SET hive.merge.size.per.task=1073741824;
SET hive.merge.orcfile.stripe.level=true;
SET hive.merge.orcfile.stripe.level.threshold=0.5;
SET hive.merge.mapfiles=true;
SET hive.merge.mapredfiles=true;
SET mapred.max.split.size=512000000;
SET hive.exec.dynamic.partition=true;
SET hive.exec.dynamic.partition.mode=nonstrict;
SET hive.enforce.bucketing=true;
SET hive.exec.max.dynamic.partitions=100000;
SET hive.exec.max.dynamic.partitions.pernode=100000;
SET hive.exec.max.created.files=100000;
SET hive.auto.convert.join=true;
SET hive.optimize.bucketmapjoin=true;
SET hive.optimize.bucketmapjoin.sortedmerge=true;
SET hive.optimize.bucketmapjoin.sortedmerge.bucketmapjoin=true;
SET hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat;
SET hive.map.aggr=true;
SET hive.merge.mapfiles=true;
SET hive.merge.mapredfiles=true;
SET hive.merge.size.per.task=256000000;
SET hive.merge.smallfiles.avgsize=16000000;
SET hive.merge.orcfile.stripe.level=true;
SET hive.merge.orcfile.stripe.level.threshold=0.5;
SET hive.merge.tezfiles=true;
SET hive.merge.smallfiles.avgsize=134217728;
SET hive.merge.size.per.task=1073741824;
SET hive.merge.orcfile.stripe.level=true;
SET hive.merge.orcfile.stripe.level.threshold=0.5;
SET hive.merge.mapfiles=true;
SET hive.merge.mapredfiles=true;
SET mapred.max.split.size=512000000;
SELECT COUNT(*) FROM db_table1
UNION ALL
SELECT COUNT(*) FROM db_table2
UNION ALL
SELECT COUNT(*) FROM db_table3
...
UNION ALL
SELECT COUNT(*) FROM db_tableN;
```
其中,需要将 `db_table1` 至 `db_tableN` 替换为实际的表名。如果不确定有哪些表名符合条件,可以使用如下的语句查询:
```sql
SHOW TABLES LIKE 'db_%';
```
该语句会列出所有表名开头为 `db_` 的表。
阅读全文