HDFS Commands - Data Engineering

HDFS commands :

HDFS commands will interact with namenode to show results
some commands like cat ,tail will interact with datanode to show results

HDFS help commands:

hadoop fs -help ls

list commands :

hadoop fs -ls /user/cloudera
hadoop fs -ls / [Sort by name]
hadoop fs -ls -t -r [sort by time in reverse order]
hadoop fs -ls -S / [sort by size by default largest is the top]
hadoop fs -ls -S -h / [ sort by size by readable format]
hadoop fs -ls -R [ list recursively]
hadoop fs -mkdir /user/cloudera/testingg
hadoop fs -mkdir -p /user/cloudera/testing1/testing2

Removing files & directories:

hadoop fs -rm /user/cloudera/testingg
hadoop fs -rm -R /user/cloudera/testingg
hadoop fs -rmdir /user/cloudera/testingg

Copying file from Local to HDFS Location:

hadoop fs -copyFromLocal desktop/file1.txt /datanode
hadoop fs - copyToLocal /datanode desktop/file.txt

hadoop fs -get desktop/file1.txt /datanode
hadoop fs -put /datanode desktop/file.txt

Reading the files :

hadoop fs -tail /user/cloudera/file1.txt
hadoop fs -cat /user/cloudera/file1.txt
hadoop fs -cp /user/cloudera/file1.txt /user/cloudera/folder2/file1.txt
hadoop fs -mv /user/cloudera/file1.txt /user/cloudera/folder2/file1.txt

File System & disk space usage:

hadoop fs -df -h /user/cloudera
hadoop fs -du -h /user/cloudera
dfs.replication=5