How to contribute limited/specific amount of storage as slave to the Hadoop cluster? 💻 💻 💻

Hola Guys ! 🤩 🤩 🤩

Hope you all are well and excited for knowing more about Hadoop Cluster. So, for answer your question I created this Blog. 😄 😄 😄

“ This Blog is for those who want to know about how create partition and then use that partition for Hadoop Cluster Datanode.”

So, let’s explore the technical stuff Together. 🤝 🤝 🤝

Prerequisite 😒 😒 😒

1. Hadoop Software should be already installed in system.
2. Java Software also should install in system.
3. There should a cluster with NameNode already configured, we will create only Datanode with limited storage.

To contribute limited/specific amount of storage as slave to the Hadoop cluster, we should follow these steps : 👇 👇 👇

1. Create Partition
2. Format the partition
3. Mount the Patition
4. Configure the Hadoop Datanode
5. Verify the Setup configuration ( Optional)

So, let’s perform above mentions steps one by one. 👣 👣 👣

How to Create Partition 📗 📗 📗

NOTE : Article contain Setup for LINUX only. 🔒🔒🔒

If you are using Virtual Machine, create a virtual volume as your requirement. Here i created 10 Gib volume disk. ( else skip this step)

Creation of Virtual Volume in Oracle VM VirtualBox 📦 📦 📦

📌 TO create volume, we have to go virtual machine setting. ⚙️

📌 Now go to storage and create new volume 🔨

📌 Now click next next to create volume as shown in screenshots. ⌛️⌛️⌛️

Step to create Virtual Volume

📌 Now our Volumes are successfully created. 💯💯💯

Now boot or turn on the Virtual machine.

Creation of the Partition by using fdisk command ♥️♥️♥️

Now we create the the partition by fdisk command. But we first list all partition.

Command : “ fdisk -l “ or “ lsblk “

Output :

fdisk command

Now we create the partition by command :

“ fdisk /dev/sdb “

Output :

Partition Creation

Now we again check partition by using “ lsblk” command.

Successfully Created

As, partition successfully created so now we move ahead and format the partition. ✔️✔️✔️

Format the Partition 🖋🖋🖋

To format the partition we first have to compare the filesystem and create format as requirement. ( here i create .ext4 format )

Command

mkfs.ext4 /dev/sdb1

Output :

Now we have successfully formatted the partition. Now last step is to mount the Drive. ✔️✔️✔️

Mount the Drive with a Directory 🔗🔗🔗

First we list all mounted directories by “df -h “ command.

Mounting the drive is very easy task. We can do this by single command, but before we have to create a directory to which we will mount the drive.

So, first create directory and then mount by using following commands :

Command:

mkdir PathToDirectory

mount /dev/sdb1 /PathToDirectory

Output :

As now disk is successfully mounted, we will verify by “df -h” command. ✔️✔️✔️

Now we configure our Datanode 🔥🔥🔥

Configuration of Datanode ⚡️⚡️⚡️

As we have Hadoop and Java already installed, so now have to follow these steps :

1. Configuration of file :

hdfs-site.xml

core-site.xml

2. Start the Datanode

3. Check Hadoop report (to verify setup)

So, first we configure the files as follows : 📋📋📋

command :

cd /etc/hadoop

vi hdfs-site.xml ( and setup file as shown in ScreenShot )

vi core-site.xml ( and setup file as shown in ScreenShot )

ScreenShots :

Commands

Files Screenshots :

Files Configuration Code

Hence, files successfully configured.

Now , we start the datanode by 🔎🔎🔎

Command :

hadoop-daemon.sh start datanode

jps ( to check datanode started or not)

At last we check the Datanode report. 📜📜📜

Command :

hadoop dfsadmin -report

Hadoop Cluster Report

Hence, we have successfully done our task. 🎯🎯🎯

That’s all ! ⭐️⭐️⭐️

So, now it’s time to say Goodbye. Now, we will meet soon , in my upcoming blog, until that be happy and safe. 🤗 🤗 🤗

If you like my blog and wants such blog follow me on medium. 👍👍👍

In upcoming days I am going to publish lots of articles on Cloud Computing Technologies and many case-study, So definitely follow me on Medium. 👀👀👀

Here is my LinkedIn profile link and if you have any queries definitely Comment. ✍️✍️✍️

Kaushal Soni - Student ( Arth - School of Technology) - ARTH - The School of Technologies |…