Search This Blog

Enabling debug logs in Apache Hadoop and Hortonworks Data Platform


To trouble shoot hadoop issues we need to have debug logs to get more low level errors.by default debug logs are not enabled in Hortonworks data platform and in plain hadoop also.
In this post,we will discuss how to enable debug logs in HDP and plain hadoop.

1. Modify /var/lib/ambari-server/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/templates/hadoop-env.sh.j2


In HDP,we need to add below line to hadoop-env.sh.j2 to enable debug logs on HDFS services.


export HADOOP_ROOT_LOGGER=DEBUG,console


If you are using Plain hadoop you can directly add above line to /etc/hadoop/conf/hadoop-env.sh file in all nodes and restart HDFS deamons on all nodes.
If you are using Hortonworks Data Platform ,You need to follow below steps.

2. Restart Ambari agents on all nodes


We need to restart ambari agent on all master nodes,data nodes and edge nodes if any. We need to run below command for restarting.

service ambari-agent restart.

Or we can stop and start ambari agent.

service ambari-agent stop
service ambari-agent start


3. Restart Ambari server


We also need to restart ambari server on first master.We can run below command.

service ambari-server restart

Or we can stop and start ambari server.

service ambari-server stop
service ambari-server start

4. Restart HDFS services


In Ambari UI ,Click on HDFS , click service actions drop down and select restart all option.It will ask for confirmation once you confirm, it will restart all HDFS daemons.

Once debug logs are enabled you can check them in name node logs and data node logs.Debug logs consume lot of space you need to disable them once you collect required logs. to disable remove added export command and restart above mentioned services.

1 comment:

  1. In common, there are more Android apps than iOS apps, however there are restrictions that have an effect on} both operating systems. Since we want to have the most effective experience possible while we’re gaming, functionality is probably the biggest consideration when it comes to of|in relation to} choosing between a cell browser and on line casino app. With this in thoughts, they have already got one foot in the door in the race to launch their online sportsbook in Maryland, with it now simply seemingly like 바카라 사이트 a matter of time before this becomes a truth. However FanDuel ended up donating $500,000 instead of $250,000, showing simply how committed they are to help sports betting launch in the state. It wouldn’t make sense for DraftKings to pledge all that money to serving to Maryland launching sports betting with out getting in on the act themselves, thus DK ought to be one of many first ones to launch in MD soon. BetRivers’ retail sportsbook can presently be found at Bingo World in Baltimore, with these in MD in a position to} place their retail bets with BetRivers for a number of|numerous|a selection of} months now following their launch.

    ReplyDelete