THM: John the Ripper: The Basics

Learn how to use John the Ripper, a powerful and adaptable hash-cracking tool.

Basic Terms

A hash is a way of taking a piece of data of any length and representing it in another fixed-length form.

This process masks the original value of the data. The hash value is obtained by running the original data through a hashing algorithm.

Many popular hashing algorithms exist, such as MD4, MD5, SHA1 and NTLM.

Hashing functions are designed as one-way functions. In other words, it is easy to calculate the hash value of a given input; however, it is a hard problem to find the original input given the hash value. In simple terms, a hard problem quickly becomes computationally infeasible in computer science. This computational problem has its roots in mathematics as P vs NP.

In computer science, P and NP are two classes of problems that help us understand the efficiency of algorithms:

P (Polynomial Time): Class P covers the problems whose solution can be found in polynomial time. Consider sorting a list in increasing order. The longer the list, the longer it would take to sort; however, the increase in time is not exponential.
NP (Non-deterministic Polynomial Time): Problems in the class NP are those for which a given solution can be checked quickly, even though finding the solution itself might be hard. In fact, we don’t know if there is a fast algorithm to find the solution in the first place.

Where John Comes in

Even though the algorithm is not feasibly reversible, that doesn’t mean cracking the hashes is impossible. If you have the hashed version of a password, for example, and you know the hashing algorithm, you can use that hashing algorithm to hash a large number of words, called a dictionary. You can then compare these hashes to the one you’re trying to crack to see if they match. If they do, you know what word corresponds to that hash- you’ve cracked it!

This process is called a dictionary attack, and John the Ripper, or John as it’s commonly shortened, is a tool for conducting fast brute force attacks on various hash types.

This room will focus on the most popular extended version of John the Ripper, Jumbo John.

Answer the questions

What is the most popular extended version of John the Ripper?

Answer: Jumbo John

Setting Up Your System

Install Jumbo John version of John the Ripper on Linux

sudo apt install john

Now that we have john ready, we must consider another indispensable component: wordlists.

There are many different wordlists out there, and a good collection can be found in the SecLists repository. There are a few places you can look for wordlists for attacking the system of choice; we will quickly run through where you can find them.

On the AttackBox and Kali Linux distributions, the /usr/share/wordlists directory contains a series of great wordlists.

For all of the tasks in this room, we will use the infamous rockyou.txt wordlist, a very large common password wordlist obtained from a data breach on a website called rockyou.com in 2009. If you are not using any of the above distributions, you can get the rockyou.txt wordlist from the SecLists repository under the /Passwords/Leaked-Databases subsection.

Answer the questions

Which website’s breach was the `rockyou.txt` wordlist created from?

Answer: rockyou.com

Cracking Basic Hashes

The basic syntax of John the Ripper commands is as follows. We will cover the specific options and modifiers used as we use them.

john [options] [file path]

john: Invokes the John the Ripper program
[options]: Specifies the options you want to use
[file path]: The file containing the hash you’re trying to crack; if it’s in the same directory, you won’t need to name a path, just the file.

For Automatic detection of the hash type you can use:

john --wordlist=[path to wordlist] [path to file]

--wordlist=: Specifies using wordlist mode, reading from the file that you supply in the provided path
[path to wordlist]: The path to the wordlist you’re using, as described in the previous task

Example Usage:

john --wordlist=/usr/share/wordlists/rockyou.txt hash_to_crack.txt

To specify a hash type:

john --format=[format] --wordlist=[path to wordlist] [path to file]

--format=: This is the flag to tell John that you’re giving it a hash of a specific format and to use the following format to crack it
[format]: The format that the hash is in

Example Usage:

john --format=raw-md5 --wordlist=/usr/share/wordlists/rockyou.txt hash_to_crack.txt

Note:

You have to prefix it with raw- to tell John you’re just dealing with a standard hash type, though this doesn’t always apply. To check if you need to add the prefix or not, you can list all of John’s formats using john --list=formats and either check manually or grep for your hash type using something like john --list=formats | grep -iF "md5".

You can use a hash identifier which is a python application called hash-id.py

wget https://gitlab.com/kalilinux/packages/hash-identifier/-/raw/kali/master/hash-id.py

python3 hash-id.py

Answer the questions

What type of hash is hash1.txt?

cat ~/John-the-Ripper-The-Basics/Task04/hash1.txt
2e728dd31fb5949bc39cac5a9f066498

Answer: md5

https://hashes.com/en/tools/hash_identifier

What is the cracked value of hash1.txt?

john --format=raw-md5 --wordlist=/usr/share/wordlists/rockyou.txt ~/John-the-Ripper-The-Basics/Task04/hash1.txt

john --show --format=raw-md5 ~/John-the-Ripper-The-Basics/Task04/hash1.txt
?:biscuit

Answer: biscuit

What type of hash is hash2.txt?

cat ~/John-the-Ripper-The-Basics/Task04/hash2.txt
1A732667F3917C0F4AA98BB13011B9090C6F8065

Answer: SHA1

https://www.tunnelsup.com/hash-analyzer/

What is the cracked value of hash2.txt?

john --format=raw-sha1 --wordlist=/usr/share/wordlists/rockyou.txt ~/John-the-Ripper-The-Basics/Task04/hash2.txt

john --show --format=raw-sha1 ~/John-the-Ripper-The-Basics/Task04/hash2.txt
?:kangeroo

Answer: kangeroo

What type of hash is hash3.txt?

cat ~/John-the-Ripper-The-Basics/Task04/hash3.txt
D7F4D3CCEE7ACD3DD7FAD3AC2BE2AAE9C44F4E9B7FB802D73136D4C53920140A

Answer: SHA256

https://hashes.com/en/tools/hash_identifier

What is the cracked value of hash3.txt?

john --format=raw-sha256 --wordlist=/usr/share/wordlists/rockyou.txt ~/John-the-Ripper-The-Basics/Task04/hash3.txt

john --show --format=raw-sha256 ~/John-the-Ripper-The-Basics/Task04/hash3.txt
?:microphone

Answer: microphone

What type of hash is hash4.txt?

cat ~/John-the-Ripper-The-Basics/Task04/hash4.txt
c5a60cc6bbba781c601c5402755ae1044bbf45b78d1183cbf2ca1c865b6c792cf3c6b87791344986c8a832a0f9ca8d0b4afd3d9421a149d57075e1b4e93f90bf

Answer: whirlpool

https://hashes.com/en/tools/hash_identifier

What is the cracked value of hash4.txt?

john --format=whirlpool --wordlist=/usr/share/wordlists/rockyou.txt ~/John-the-Ripper-The-Basics/Task04/hash4.txt

john --show --format=whirlpool ~/John-the-Ripper-The-Basics/Task04/hash4.txt
?:colossal

Answer: colossal

Cracking Windows Authentication Hashes

Authentication hashes are the hashed versions of passwords stored by operating systems; it is sometimes possible to crack them using our brute-force methods.

In Windows, SAM (Security Account Manager) is used to store user account information, including usernames and hashed passwords. You can acquire NTHash/NTLM hashes by dumping the SAM database on a Windows machine, using a tool like Mimikatz, or using the Active Directory database: NTDS.dit.

Answer the questions

What do we need to set the `--format` flag to in order to crack this hash?

john --list=formats | grep -iF 'ntlm'

Answer: nt

What is the cracked value of this password?

john --format=NT --wordlist=/usr/share/wordlists/rockyou.txt ~/John-the-Ripper-The-Basics/Task05/ntlm.txt

Answer: mushroom

Cracking /etc/shadow Hashes

John can be very particular about the formats it needs data in to be able to work with it; for this reason, to crack /etc/shadow passwords, you must combine it with the /etc/passwd file for John to understand the data it’s being given. To do this, we use a tool built into the John suite of tools called unshadow. The basic syntax of unshadow is as follows:

unshadow [path to passwd] [path to shadow]

unshadow: Invokes the unshadow tool
[path to passwd]: The file that contains the copy of the /etc/passwd file you’ve taken from the target machine

root:x:0:0::/root:/bin/bash

[path to shadow]: The file that contains the copy of the /etc/shadow file you’ve taken from the target machine root:$6$2nwjN454g.dv4HN/$m9Z/r2xVfweYVkrr.v5Ft8Ws3/YYksfNwq96UL1FX0OJjY1L6l.DS3KEVsZ9rOVLB/ldTeEL/OIhJZ4GMFMGA0:18576::::::

Example Usage:

unshadow local_passwd local_shadow > unshadowed.txt

Then you can crack the password:

john --wordlist=/usr/share/wordlists/rockyou.txt --format=sha512crypt unshadowed.txt

Answer the questions

What is the root password?

john --wordlist=/usr/share/wordlists/rockyou.txt --format=sha512crypt ~/John-the-Ripper-The-Basics/Task06/etc_hashes.txt

Answer: 1234

Single Crack Mode

In this mode, John uses only the information provided in the username to try and work out possible passwords heuristically by slightly changing the letters and numbers contained within the username.

Word Mangling

Consider the username “Markus”.

Some possible passwords could be:

Markus1, Markus2, Markus3 (etc.)
MArkus, MARkus, MARKus (etc.)
Markus!, Markus$, Markus* (etc.)

GECOS stands for General Electric Comprehensive Operating System. In the last task, we looked at the entries for both /etc/shadow and /etc/passwd.

Looking closely, you will notice that the fields are separated by a colon :.

The fifth field in the user account record is the GECOS field.

It stores general information about the user, such as the user’s full name, office number, and telephone number, among other things.

John can take information stored in those records, such as full name and home directory name, to add to the wordlist it generates when cracking /etc/shadow hashes with single crack mode.

john --single --format=[format] [path to file]

--single: This flag lets John know you want to use the single hash-cracking mode
--format=[format]: As always, it is vital to identify the proper format.

If you’re cracking hashes in single crack mode, you need to change the file format that you’re feeding John for it to understand what data to create a wordlist from. You do this by prepending the hash with the username that the hash belongs to, so according to the above example, we would change the file `hashes.txt`

From 1efee03cdcb96d90ad48ccc7b8666033

To mike:1efee03cdcb96d90ad48ccc7b8666033

Answer the questions

What is the root password?

cat ~/John-the-Ripper-The-Basics/Task07/hash07.txt
7bf6d9bb82bed1302f331fc6b816aada
vi ~/John-the-Ripper-The-Basics/Task07/hash07.txt
cat ~/John-the-Ripper-The-Basics/Task07/hash07.txt
joker:7bf6d9bb82bed1302f331fc6b816aada

john --single --format=raw-md5 ~/John-the-Ripper-The-Basics/Task07/hash07.txt

Answer: Jok3r

Custom Rules

Custom rules are defined in the john.conf file. This file can be found in /opt/john/john.conf on the TryHackMe Attackbox. It is usually located in /etc/john/john.conf if you have installed John using a package manager or built from source with make.

[List.Rules:THMRules] is used to define the name of your rule; this is what you will use to call your custom rule a John argument.

Az: Takes the word and appends it with the characters you define
A0: Takes the word and prepends it with the characters you define
c: Capitalises the character positionally

Lastly, we must define what characters should be appended, prepended or otherwise included. We do this by adding character sets in square brackets [ ] where they should be used. These follow the modifier patterns inside double quotes " "

[0-9]: Will include numbers 0-9
[0]: Will include only the number 0
[A-z]: Will include both upper and lowercase
[A-Z]: Will include only uppercase letters
[a-z]: Will include only lowercase letters

[a]: Will include only a
[!£$%@]: Will include the symbols !, £, $, %, and @

Example:

[List.Rules:PoloPassword]

cAz"[0-9] [!£$%@]"

Utilises the following:

c: Capitalises the first letter
Az: Appends to the end of the word
[0-9]: A number in the range 0-9
[!£$%@]: The password is followed by one of these symbols

We could then call this custom rule a John argument using the --rule=PoloPassword flag.

john --wordlist=[path to wordlist] --rule=PoloPassword [path to file]

Jumbo John already has an extensive list of custom rules containing modifiers for use in almost all cases. If you get stuck, try looking at those rules [around line 678] if your syntax isn’t working correctly.

Answer the questions

What do custom rules allow us to exploit?

Answer: Password complexity predictability

What rule would we use to add all capital letters to the end of the word?

Answer: Az"[A-Z]"

What flag would we use to call a custom rule called `THMRules`?

Answer: --rule=THMRules

Cracking Password Protected Zip Files

we will use the zip2john tool to convert the Zip file into a hash format that John can understand and hopefully crack. The primary usage is like this:

zip2john [options] [zip file] > [output file]

[options]: Allows you to pass specific checksum options to zip2john; this shouldn’t often be necessary
[zip file]: The path to the Zip file you wish to get the hash of
>: This redirects the output from this command to another file
[output file]: This is the file that will store the output

Example Usage

zip2john zipfile.zip > zip_hash.txt

We’re then able to take the file we output from zip2john in our example use case, zip_hash.txt, and, as we did with unshadow, feed it directly into John as we have made the input specifically for it.

john --wordlist=/usr/share/wordlists/rockyou.txt zip_hash.txt

Answer the questions

What is the password for the secure.zip file?

zip2john ~/John-the-Ripper-The-Basics/Task09/secure.zip > ~/John-the-Ripper-The-Basics/Task09/secure.zip.txt

cat ~/John-the-Ripper-The-Basics/Task09/secure.zip.txt 
secure.zip/zippy/flag.txt:$pkzip$1*2*2*0*26*1a*849ab5a6*0*48*0*26*b689*964fa5a31f8cefe8e6b3456b578d66a08489def78128450ccf07c28dfa6c197fd148f696e3a2*$/pkzip$:zippy/flag.txt:secure.zip::/home/user/John-the-Ripper-The-Basics/Task09/secure.zip

john --wordlist=/usr/share/wordlists/rockyou.txt ~/John-the-Ripper-The-Basics/Task09/secure.zip.txt

Answer: pass123

What is the contents of the flag inside the zip file?

unzip ~/John-the-Ripper-The-Basics/Task09/secure.zip
Archive:  /home/user/John-the-Ripper-The-Basics/Task09/secure.zip
[/home/user/John-the-Ripper-The-Basics/Task09/secure.zip] zippy/flag.txt password: 
 extracting: zippy/flag.txt  
 
cat zippy/flag.txt
THM{w3ll_d0n3_h4sh_r0y4l}

Answer: THM{w3ll_d0n3_h4sh_r0y4l}

Cracking Password Protected RAR archive

Almost identical to the zip2john tool, we will use the rar2john tool to convert the RAR file into a hash format that John can understand. The basic syntax is as follows:

rar2john [rar file] > [output file]

rar2john: Invokes the rar2john tool
[rar file]: The path to the RAR file you wish to get the hash of
>: This redirects the output of this command to another file
[output file]: This is the file that will store the output from the command

Example Usage

/opt/john/rar2john rarfile.rar > rar_hash.txt

we can take the file we output from rar2john in our example use case, rar_hash.txt, and feed it directly into John as we did with zip2john.

john --wordlist=/usr/share/wordlists/rockyou.txt rar_hash.txt

Answer the questions

What is the password for the secure.rar file?

rar2john ~/John-the-Ripper-The-Basics/Task10/secure.rar > ~/John-the-Ripper-The-Basics/Task10/secure.rar.txt

cat ~/John-the-Ripper-The-Basics/Task10/secure.rar.txt 
secure.rar:$rar5$16$b7b0ffc959b2bc55ffb712fc0293159b$15$4f7de6eb8d17078f4b3c0ce650de32ff$8$ebd10bb79dbfb9f8

john --wordlist=/usr/share/wordlists/rockyou.txt ~/John-the-Ripper-The-Basics/Task10/secure.rar.txt

Answer: password

What are the contents of the flag inside the zip file?

unrar x ~/John-the-Ripper-The-Basics/Task10/secure.rar 
 
cat flag.txt 
THM{r4r_4rch1ve5_th15_t1m3}

Answer: THM{r4r_4rch1ve5_th15_t1m3}

Cracking SSH Keys with John

As the name suggests, ssh2john converts the id_rsa private key, which is used to log in to the SSH session, into a hash format that John can work with. Jokes aside, it’s another beautiful example of John’s versatility. The syntax is about what you’d expect. Note that if you don’t have ssh2john installed, you can use ssh2john.py, located in the /opt/john/ssh2john.py. If you’re doing this on the AttackBox, replace the ssh2john command with python3 /opt/john/ssh2john.py or on Kali, python /usr/share/john/ssh2john.py.

ssh2john [id_rsa private key file] > [output file]

ssh2john: Invokes the ssh2john tool
[id_rsa private key file]: The path to the id_rsa file you wish to get the hash of
>: This is the output director. We’re using it to redirect the output from this command to another file.
[output file]: This is the file that will store the output from

Example Usage

/opt/john/ssh2john.py id_rsa > id_rsa_hash.txt

we’re feeding the file we output from ssh2john, which in our example use case is called id_rsa_hash.txt and, as we did with rar2john, we can use this seamlessly with John:

john --wordlist=/usr/share/wordlists/rockyou.txt id_rsa_hash.txt

Answer the questions

What is the password for the secure.rar file?

ssh2john.py ~/John-the-Ripper-The-Basics/Task11/id_rsa > ~/John-the-Ripper-The-Basics/Task11/id_rsa.txt

john --wordlist=/usr/share/wordlists/rockyou.txt ~/John-the-Ripper-The-Basics/Task11/id_rsa.txt

Answer: mango

I’d recommend checking out the Openwall Wiki here for more information about using John and advice, updates, or news about the tool.