an error occurred when build a sample cube at step 5:create HTable #470
Comments
|
It seems the new table KYLIN_XSLUXA5KGI was not fully successfully created. You will want to recover the new table to a good state (as well as HBase itself), and then resume job in Kylin. You have found the new regions were in transition, which is a good clue. It seems the new table was assigned to some unstable region server, so the new regions had to be transferred quickly after the table was created. And leave the table in an unavailable state. HBase master log is where to find more info about the unstable region server. Identify the unstable region server, fix it, or remove it from cluster to ensure it won't take new regions again. Verify the HBase cluster by creating new table from hbase shell and scanning it. Finally drop table KYLIN_XSLUXA5KGI and resume job in Kylin. Step 5 will recreated the table anyway, the old table is not needed. |
|
Btw, next time pls raise questions in [email protected] -- there you find all the Kylin community. This KylinOLAP git repo is deprecated. |
|
tks,liyang,i will send issue to [email protected] next time. BTW, i resolve the problem . i check my hdfs log and find some information below:
then i realized probably my config about Compression algorithm is wrong To disable compressing MR jobs you need to modify $KYLIN_HOME/conf/kylin_job_conf.xml by removing all configuration entries related to compression(Just grep the keyword “compress”). To disable compressing hbase tables you need to open $KYLIN_HOME/conf/kylin.properties and remove the line starting with kylin.hbase.default.compression.codec. finally, i restart my service and it works well. |
|
Thanks for sharing the solution. Others will learn from it. On Tue, Apr 12, 2016 at 3:54 PM, yeqing0907 [email protected]
|
|
赞！ |
|
open $KYLIN_HOME/conf/kylin.properties and remove the line starting with kylin.hbase.default.compression.codec. |
env:
hadoop 2.7.1
hbase 1.1.4
hive 2.0.0
kylin-1.5.0-HBase1.1.3(special version)
I an login in the web ui:192.168.10.20:7070/kylin with ADMIN/KYLIN
when i select action Build and the select a data then submit the job
the job is submit successful
then a change the tab to monitor see how it works
step 1 - 4 is working well
but when it comes step 5:create Htable ,it throws a problem :
(我能正常运行kylin项目并且能正常访问，我用sample.sh做测试，然后试图执行一个build操作，选择了一个日期以后。我提交了任务，能正常提交，然后我进入了monitor去看执行进度，然后第1-4步都能够成功执行，到第五步创建Htable的时候报错了，看了下log 提示的是 result code ：2 。看了控制台kylin.log的日志，报错如下：)
2016-04-08 17:11:24,207 DEBUG [pool-7-thread-5] service.CubeService:578 : on updateOnNewSegmentReady: kylin_sales_cube
2016-04-08 17:11:24,207 DEBUG [pool-7-thread-5] service.CubeService:581 : server mode: all
2016-04-08 17:11:24,209 INFO [http-bio-7070-exec-8] controller.CacheController:64 : wipe cache type: CUBE event:UPDATE name:kylin_sales_cube
2016-04-08 17:11:24,210 INFO [http-bio-7070-exec-8] service.CacheService:169 : rebuild cache type: CUBE name:kylin_sales_cube
2016-04-08 17:11:24,216 DEBUG [http-bio-7070-exec-8] cube.CubeManager:840 : Reloaded new cube: kylin_sales_cube with reference beingCUBE[name=kylin_sales_cube] having 1 segments:KYLIN_XSLUXA5KGI
2016-04-08 17:11:24,221 INFO [http-bio-7070-exec-8] service.CacheService:122 : removeOLAPDataSource is called for project learn_kylin
2016-04-08 17:11:24,221 WARN [http-bio-7070-exec-8] service.CacheService:108 : skip cleaning cache for 2fbca32a-a33e-4b69-83dd-0bb8b1f8c53b
2016-04-08 17:11:24,222 DEBUG [http-bio-7070-exec-8] service.CubeService:578 : on updateOnNewSegmentReady: kylin_sales_cube
2016-04-08 17:11:24,222 DEBUG [http-bio-7070-exec-8] service.CubeService:581 : server mode: all
2016-04-08 17:11:24,630 INFO [pool-7-thread-5] steps.CubeHTableUtil:92 : hbase will use snappy to compress data
2016-04-08 17:11:24,632 INFO [pool-7-thread-5] steps.CubeHTableUtil:92 : hbase will use snappy to compress data
2016-04-08 17:11:24,671 INFO [pool-7-thread-5] util.DeployCoprocessorCLI:180 : Add coprocessor on KYLIN_XSLUXA5KGI
2016-04-08 17:11:24,673 INFO [pool-7-thread-5] util.DeployCoprocessorCLI:160 : hbase table KYLIN_XSLUXA5KGI deployed with coprocessor.
2016-04-08 17:11:29,217 ERROR [pool-7-thread-5] steps.CubeHTableUtil:135 : Failed to create HTable
java.lang.IllegalArgumentException: table KYLIN_XSLUXA5KGI created, but is not available due to some reasons
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92)
at org.apache.kylin.storage.hbase.steps.CubeHTableUtil.createHTable(CubeHTableUtil.java:132)
at org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:60)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-04-08 17:11:29,228 ERROR [pool-7-thread-5] steps.CreateHTableJob:109 : table KYLIN_XSLUXA5KGI created, but is not available due to some reasons
java.lang.IllegalArgumentException: table KYLIN_XSLUXA5KGI created, but is not available due to some reasons
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92)
at org.apache.kylin.storage.hbase.steps.CubeHTableUtil.createHTable(CubeHTableUtil.java:132)
at org.apache.kylin.storage.hbase.steps.CreateHTableJob.run(CreateHTableJob.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:60)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-04-08 17:11:29,246 DEBUG [pool-7-thread-5] hbase.HBaseResourceStore:277 : Update row /execute_output/4020e441-a770-4200-b1c3-c8da3ead190c-04 from oldTs: 1460106683759, to newTs: 1460106689234, operation result: true
2016-04-08 17:11:29,263 DEBUG [pool-7-thread-5] hbase.HBaseResourceStore:277 : Update row /execute_output/4020e441-a770-4200-b1c3-c8da3ead190c-04 from oldTs: 1460106689234, to newTs: 1460106689254, operation result: true
2016-04-08 17:11:29,264 INFO [pool-7-thread-5] manager.ExecutableManager:273 : job id:4020e441-a770-4200-b1c3-c8da3ead190c-04 from RUNNING to ERROR
it says : "table KYLIN_XSLUXA5KGI created, but is not available due to some reasons"
but don't say why.
then i locate the source jar to find where the hint from
in kylin-coprocessor-1.5.0-SNAPSHOT.jar | org.apache.kylin.storage.hbase.steps.CubeHTableUtil.java
line 131 and 132 :
admin.createTable(tableDesc, splitKeys);
Preconditions.checkArgument(admin.isTableAvailable(TableName.valueOf(tableName)), "table " + tableName + " created, but is not available due to some reasons");
so, as you see , the table which created in hbase is not avilable.
（通过报错信息，我去找源代码试图找出错误原因，但是源代码里面给的信息也不是很明确，说的是这个表不是可用的，但是为什么不可用却没说）
and when i check my hbase web console , i find the table data is not on any region
in Task Tab like this:
the table have a task seems like closing regin
give some table info .hope it can give you some information
（通过查看hbase web控制台 我发现这张表的信息如下：
'KYLIN_XSLUXA5KGI', {TABLE_ATTRIBUTES => {coprocessor$1 => 'hdfs://master1:9000/home/hadoop/kylin/kylin_hdfs_working_dir/kylin_metadata/coprocessor/kylin-coprocessor-1.5.0-SNAPSHOT-0.jar|org.apache.kylin.storage.hbase.ii.coprocessor.endpoint.IIEndpoint|1000|', coprocessor$2 => 'hdfs://master1:9000/home/hadoop/kylin/kylin_hdfs_working_dir/kylin_metadata/coprocessor/kylin-coprocessor-1.5.0-SNAPSHOT-0.jar|org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService|1001|', coprocessor$3 => 'hdfs://master1:9000/home/hadoop/kylin/kylin_hdfs_working_dir/kylin_metadata/coprocessor/kylin-coprocessor-1.5.0-SNAPSHOT-0.jar|org.apache.kylin.storage.hbase.cube.v1.coprocessor.observer.AggregateRegionObserver|1002|', METADATA => {'CREATION_TIME' => '1460106684505', 'KYLIN_HOST' => 'kylin_metadata', 'OWNER' => '[email protected]', 'SPLIT_POLICY' => 'org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy'}}, {NAME => 'F1', DATA_BLOCK_ENCODING => 'FAST_DIFF', COMPRESSION => 'SNAPPY', BLOCKSIZE => '4194304'}, {NAME => 'F2', DATA_BLOCK_ENCODING => 'FAST_DIFF', COMPRESSION => 'SNAPPY', BLOCKSIZE => '4194304'}
会不会是协处理器的问题？
）
然后发现在build以后，生成的表没有在任何region上面
还执行了closing region 的任务
Closing region KYLIN_XSLUXA5KGI,\x00\x01,1460106684846.971eccb68fa9ce8116596ea73cc9ebb0.
Name ：KYLIN_XSLUXA5KGI,,1460106684846.fe79ee4e998f5816b7d9639d8313fba6.
Region Server ：not deployed
这是不是表示数据没有写入任何的region中？
at last , my kylin.properties is like this:
kylin.owner=[email protected]
kylin.rest.servers=192.168.10.10:7070(master1 is my namenode ip,also running zookeeper)
kylin.metadata.url=kylin_metadata@hbase
kylin.storage.url=hbase
kylin.hdfs.working.dir=/home/hadoop/kylin/kylin_hdfs_working_dir
kylin.job.jar=/home/hadoop/kylin/lib/kylin-job-1.5.0-SNAPSHOT.jar
kylin.coprocessor.local.jar=/home/hadoop/kylin/lib/kylin-coprocessor-1.5.0-SNAPSHOT.jar
kylin.job.mapreduce.default.reduce.input.mb=500
kylin.job.remote.cli.working.dir=/home/hadoop/kylin/kylin_job_remote_cli_working_dir
in file kylin_job and kylin_hive , i just changed the replication "4" and compression "fasle"
please help me ,thanks
regards
a hopeless gilr from qingdao.shandong.china
（最后，我把我的配置信息贴出来，希望你们帮帮我 —— 一个无助的女程序员）
^ ^
The text was updated successfully, but these errors were encountered: