hive解决数据倾斜代码

以下是一些解决Hive数据倾斜的代码示例：

1. 随机数打散：

set hive.exec.reducers.bytes.per.reducer=256000000;
set hive.exec.reducers.max=99999;

select /*+ mapjoin(b) */ a.id, a.name, b.salary
from table_a a join (
  select rand() as r, salary from table_b distribute by rand() sort by r
) b on a.id = b.id;

2. 增加reduce任务数：

set mapreduce.job.reduces=100;

select ...
from ...
distribute by rand();

3. 动态分区：

set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;

insert overwrite table table_c partition(dt)
select id, name, salary, dt
from table_a
distribute by id;

4. 压缩数据：

set mapreduce.map.output.compress=true;
set mapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;

select ...
from ...
distribute by rand();

以上代码示例是针对不同场景的解决方案，希望这些代码可以为你解决Hive数据倾斜问题提供一些帮助。