Data Engineering: Embracing the UNIX philosophy - Part 2

Published

July 12, 2023

Slides

The slides for today’s lesson are available online as an PDF file. Use the button below to open the slides either in a new tab in your browser. Or you can just scroll the embedded slides below. For the embedded slides below, you can also click in the slides and navigate through them with your left and right arrow keys, for convenience.

Download PDF of all slides

Creating the ninja-way.csv file manually

You can use the following instructions at the bash command line in your desired working directory to create, i.e., reproduce the ninja-way.csv file manually.

In order to create the ninja-way.csv file as used in these lecture notes, you will need to perform the following steps in a bash terminal session:

  1. First run cat > ninja-way.csv
  2. The prompt will move to the next line. So cat is waiting for input from you from stdin, i.e., your keyboard!
  3. Copy paste the following text into the terminal, where the prompt is active.
This is a nice csv containing characters from the Anime: Naruto
This is based on a manga by various authors
See the following fields which contain the data
id,first_name,last_name,village,season_first_appearance,home
1,Naruto,Uzamaki,leaves,1,leaves village
1,Naruto,Uzamaki,leaves,1,leaves village
1,Naruto,Uzamaki,leaves,1,leaves village
2,Sasuke,Uchiha,leaves,1,leaves village


3,Sakura,Haruno,leaves,1,leaves village
TODO: add more leaves village characters

4,Gaara,None,sand,2,sand village
4,Gaara,None,sand,2,sand village
5,Temari,Nara,sand,2,sand village



## we should add more sand village characters

6,Sai,Yamanaka,leaves,4,leaves village

#closing the file now
  1. Make sure that you hit the Return key after the last pasted line. This will insert a blank line after the pasting is complete. This is important to do else you will end up omitting the final commented line in the above pasted text.
  2. You can now hit Ctrl-C to terminate the cat input command. The ninja-way.csv file should now be created.
  3. Run less ninja-way.csv to view the newly created file and check that you get the same contents as those that you just pasted in Step 3.