pinpoint hbase占用空间过大清理

[root@XX-001 bin]# du -h –max-depth=1 /data
375G    /data/service
16K    /data/lost+found
1.1G    /data/soft
376G    /data
[root@XX-001 bin]# du -h –max-depth=1 /data/service/
63M    /data/service/zookeeper
1.6G    /data/service/tomcat-web
370G    /data/service/hbase
257M    /data/service/tomcat-col
3.0G    /data/service/pinpoint
375G    /data/service/
[root@XX-001 bin]# du -h –max-depth=1 /data/service/hbase/
367G    /data/service/hbase/data
3.5G    /data/service/hbase/logs
204K    /data/service/hbase/bin
80M    /data/service/hbase/lib
672K    /data/service/hbase/hbase-webapps
44K    /data/service/hbase/conf
277M    /data/service/hbase/docs
370G    /data/service/hbase/

 

[root@XX-001 default]# du -h –max-depth=1 ./
33M    ./HostApplicationMap_Ver2
1.1M    ./ApplicationStatAggre
229M    ./ApplicationMapStatisticsCaller_Ver2
32M    ./AgentLifeCycle
163M    ./ApplicationMapStatisticsCallee_Ver2
42M    ./ApplicationMapStatisticsSelf_Ver2
353G    ./TraceV2                          ————这个表最大
7.2G    ./ApplicationTraceIndex
44K    ./ApplicationIndex
11M    ./AgentInfo
3.1M    ./StringMetaData
27M    ./AgentEvent
8.0M    ./SqlMetaData_Ver2
2.7G    ./AgentStatV2
119M    ./ApiMetaData
364G    ./

所以,完整的操作应该是这样的(使用的CDH):
1、进入hbase shell,查看具体表desc ‘tablename’
2、停用表disable ‘tablename’
3、更改表的TTL(time to live)值(一个int值,单位是秒)alter ‘tablename’,{NAME=>’columnName’, TTL => ‘604800’},这就表示这个表的数据只保存七天
4、enable ‘tablename’重新启用表——–这里千万不要退出不然就重用不了了。因为zk认为是起来的
此时,如果查看df -lh,你会发现数据还没有被删除,因为空间还没有释放,但是在hbase shell之中你已经查询不到这些数据了。
5.major_compact(‘tablename’)  需要进行mojor 空间才会释放

查看表结枸 打开界面即可查看 http://pp:16010/master-status#userTables

‘TraceV2’, {NAME => ‘S’, TTL => ‘5184000 SECONDS (60 DAYS)’}

命令查看   现在是保留60天改成7天
hbase(main):004:0> desc ‘TraceV2’
Table TraceV2 is ENABLED                                                                                                       
TraceV2                                                                                                                        
COLUMN FAMILIES DESCRIPTION                                                                                                    
{NAME => ‘S’, BLOOMFILTER => ‘ROW’, VERSIONS => ‘1’, IN_MEMORY => ‘false’, KEEP_DELETED_CELLS => ‘FALSE’, DATA_BLOCK_ENCODING =>
‘NONE’, TTL => ‘5184000 SECONDS (60 DAYS)’, COMPRESSION => ‘NONE’, MIN_VERSIONS => ‘0’, BLOCKCACHE => ‘true’, BLOCKSIZE => ‘655
36′, REPLICATION_SCOPE => ‘0’}                                                                                                 
1 row(s) in 0.2930 seconds

 

[root@XX-001 bin]# ./hbase shell
desc ‘TraceV2’
disable ‘TraceV2’
alter ‘TraceV2′,{NAME=>’S’, TTL => ‘604800’}
enable ‘TraceV2’                              ————–400G 差不多5分钟
major_compact(‘TraceV2’)

结果
hbase(main):005:0* major_compact(‘TraceV2’)
0 row(s) in 1.5340 seconds

发表评论