Using subprocess Popen

A coworker is proficient in Bash tells you that most data engineering tasks should be done in the shell. You mention a scripting language like Python can build robust production systems that have high quality. The code is often easier in practice to write and maintain, even if you are directly calling shell commands. You demonstrate how this works using a small Python script that you write that finds all of the Python packages installed.

Use Python, subprocess.Popen(), and the bash command pip list --format=json command, to find all of the installed packages. The pip tool is a common method of installing Python packages. The result will be a byte string, a Python3 construct. The Popen command will use PIPE to send the JSON output to stdout.

Use the with context manager to run subprocess.Popen().
Pipe the output of subprocess.Popen() to stdout as an iterator.
Convert the JSON payload to a Python dictionary with json.loads after extracting the only element of the results list.
Print the result using the pprint function of the pprint package.

Exercise

Using subprocess Popen

Instructions