Book Review: "Python for Secret Agents"
'Python for Secret Agents' by Steven F. Lott, published by Packt and sold through O'Reilly and Amazon, is easily worth $10 - $25 for any novice python programmer. The book is roughly 200 pages of python data manipulation tricks, from data generators and loops to parsing complex structures. The book has a wide collection of fundamental data manipulation techniques, but lacks more offensive techniques, such as those covered in 'Violent Python' and 'Black Hat Python'. It does however follow a consistently amusing theme, of accomplishing basic data manipulation tasks as a secret agent preparing data for HQ. Again, the tasks aren't actually security related, which makes the whole theme feel a little forced and cheesy, none the less it is an entertaining take on a programing book while still conveying fundamental python features structures. It's also important to note that the entire book deals strictly with python 3, which is not compatible with python2.7 (the python version I mainly refer to throughout this blog). All of that said, I give this book a solid 5/10 stars, as I think it's more of a beginner to intermediate general python book, and has little to do with security programing or things related to secret agents (other than its cute storyline). I also recommend this book to those looking to get better with programming and web programmers, but don't really recommend this book for penetration testers or security engineers in particular. The following is my typical table of contents overview format:
Chapter 1: Our Espionage Toolkit (Setup and iIntro)
Getting the tools of the trade - Python 3.3
Windows secrets (dependencies)
Mac OS X secrets (dependencies)
Confirming our tools
How do we stop?
Using the help() system
Mac OS and GNU/Linux secrets (dependencies)
Windows secrets (dependencies)
Using the help mode
Background briefing - math and numbers
The usual culprits
The ivory tower of numbers
Integer numbers
Rational numbers
Floating-point numbers
Decimal numbers
Complex numbers
Outside the numbers
Assigning value to variables
Writing scripts and seeing output
Gather user input
Handling exceptions
Looping and trying again
Handling text and strings
Converting between numbers and strings
Parsing strings
Organizing our software
Working with files and folders
Creating a file
Reading a file
Defining more complex logical conditions
Summary
Chapter 2: Acquiring Intelligence Data
Accessing data from the Internet
Background briefing - the TCP/IP protocols
Using http.client for HTTP GET
Changing our client information
Using FTP in Python
Downloading a file via FTP
Using FTP get() function
Using urllib for HTTP, FTP, or file access
Using urllib for FTP access
Using a REST API in Python
Getting simple REST data
Using more complex RESTful queries
Saving our data via JSON
Organizing collections of data
Using a python list
Using list index operations
Using a python tuple
Using generator expressions with list of tuples
Using a Python dictionary mapping
Using the dictionary access methods
Transforming sequences with generator functions
Using the defaultdict and counter mappings
Using a Python set
Using the for statement with a collection
Using Python operators on collections
Solving problems - currency conversion rates
Summary
Chapter 4: Encoding Secret Messages with Stenanography
Background briefing - handling file formats
Working with the OS filesystem
glob
os
Processing simple text files
Working with ZIP files
Working with JSON files
Working with CSV files
JPEG and PNG graphics - pixels and metadata
Using the Pillow library
Adding the required supporting libraries
GNU/Linux secrets
Mac OS X secrets
Windows secrets
Installing and confirming Pillow
Decoding and encoding image data
Manipulating images - resizing and thumbnails
Manipulating images - cropping
Manipulating images - enhancing
Manipulating images - filtering
Manipulating images - ImageOps
Some approaches to steganography
Getting the red-channel data
Extracting bytes from Unicode characters
Manipulating bits and bytes
Encoding the message
Decoding a message
Detecting and preventing tampering
Using hash totals to validate a file
Using a key with a digest
Solving problems - encrypting a message
Unpacking a message
Summary
Chapter 4: Drops, Hideouts, Meetups, and Lairs
Background briefing - latitude, longitude, and GPS
Coping with GPS device limitations
Handling politics - borders, precincts, jurisdictions, and neighborhoods
Finding out where we are with geocoding services
Geocoding an address
Reverse geocoding a latitude-longitude point
How close? What direction?
Combining geocoding and haversine
Compressing data to make grid codes
Creating GeoRef codes
Decoding a GeoRef code
Creating Maidenhead grid codes
Creating natural area codes
Decoding natural area codes
Solving problems - closest good restaurant
Creating simple Python objects
Working with HTML web services - tools
Working with HTML web services - getting the page
Working with HTML web services - parsing a table
Making a simple Python object from columns of data
Enriching Python objects with geocodes
Enriching Python objects with heath scores
Combining the pieces and parts
Working with clean data portals
Making a simple Python object from a JSON document
Combining different pieces and parts
Final steps
Understanding the data - schema and metadata
Summary
Chapter 5: A Spymaster's More Sensitive Analyses
Creating statistical summaries
Parsing the raw data file
Finding an average value
Understanding generator expressions
Finding the value in the middle
Finding the most popular value
Creating Python modules and applications
Creating and using a module
Creating an application module
Creating a hybrid module
Creating our own classes of objects
Using a class definition
Comparisons and correlations
Computing the standard deviation
Computing a standardized score
Comparing a sequence and an iterable
Computing a coefficient of correlation
Writing high-quality software
Building a self-testing module and a test module
Creating more sophisticated tests
Adding doctest cases to a class definition
Solving problems - analyzing some interesting datasets
Getting some more data
Further research
Summary
Index
There you have it. You can easily tell that the sub-chapters and topics are all basic python data manipulation based, as opposed to security examples. This, coupled with the fact that the code is generally not compatible with python 2.7 code, makes this book for the narrower audience of those wanting to lean general programming vs those looking to learn new security related techniques. That said, the book keeps the reader entertained with its 'secret agent' them. Don't take my word for it, check out the preview offered by O'Reilly, and dig into some python 3 web queries and data parsing.
Chapter 1: Our Espionage Toolkit (Setup and iIntro)
Getting the tools of the trade - Python 3.3
Windows secrets (dependencies)
Mac OS X secrets (dependencies)
Confirming our tools
How do we stop?
Using the help() system
Mac OS and GNU/Linux secrets (dependencies)
Windows secrets (dependencies)
Using the help mode
Background briefing - math and numbers
The usual culprits
The ivory tower of numbers
Integer numbers
Rational numbers
Floating-point numbers
Decimal numbers
Complex numbers
Outside the numbers
Assigning value to variables
Writing scripts and seeing output
Gather user input
Handling exceptions
Looping and trying again
Handling text and strings
Converting between numbers and strings
Parsing strings
Organizing our software
Working with files and folders
Creating a file
Reading a file
Defining more complex logical conditions
Summary
Chapter 2: Acquiring Intelligence Data
Accessing data from the Internet
Background briefing - the TCP/IP protocols
Using http.client for HTTP GET
Changing our client information
Using FTP in Python
Downloading a file via FTP
Using FTP get() function
Using urllib for HTTP, FTP, or file access
Using urllib for FTP access
Using a REST API in Python
Getting simple REST data
Using more complex RESTful queries
Saving our data via JSON
Organizing collections of data
Using a python list
Using list index operations
Using a python tuple
Using generator expressions with list of tuples
Using a Python dictionary mapping
Using the dictionary access methods
Transforming sequences with generator functions
Using the defaultdict and counter mappings
Using a Python set
Using the for statement with a collection
Using Python operators on collections
Solving problems - currency conversion rates
Summary
Chapter 4: Encoding Secret Messages with Stenanography
Background briefing - handling file formats
Working with the OS filesystem
glob
os
Processing simple text files
Working with ZIP files
Working with JSON files
Working with CSV files
JPEG and PNG graphics - pixels and metadata
Using the Pillow library
Adding the required supporting libraries
GNU/Linux secrets
Mac OS X secrets
Windows secrets
Installing and confirming Pillow
Decoding and encoding image data
Manipulating images - resizing and thumbnails
Manipulating images - cropping
Manipulating images - enhancing
Manipulating images - filtering
Manipulating images - ImageOps
Some approaches to steganography
Getting the red-channel data
Extracting bytes from Unicode characters
Manipulating bits and bytes
Encoding the message
Decoding a message
Detecting and preventing tampering
Using hash totals to validate a file
Using a key with a digest
Solving problems - encrypting a message
Unpacking a message
Summary
Chapter 4: Drops, Hideouts, Meetups, and Lairs
Background briefing - latitude, longitude, and GPS
Coping with GPS device limitations
Handling politics - borders, precincts, jurisdictions, and neighborhoods
Finding out where we are with geocoding services
Geocoding an address
Reverse geocoding a latitude-longitude point
How close? What direction?
Combining geocoding and haversine
Compressing data to make grid codes
Creating GeoRef codes
Decoding a GeoRef code
Creating Maidenhead grid codes
Creating natural area codes
Decoding natural area codes
Solving problems - closest good restaurant
Creating simple Python objects
Working with HTML web services - tools
Working with HTML web services - getting the page
Working with HTML web services - parsing a table
Making a simple Python object from columns of data
Enriching Python objects with geocodes
Enriching Python objects with heath scores
Combining the pieces and parts
Working with clean data portals
Making a simple Python object from a JSON document
Combining different pieces and parts
Final steps
Understanding the data - schema and metadata
Summary
Chapter 5: A Spymaster's More Sensitive Analyses
Creating statistical summaries
Parsing the raw data file
Finding an average value
Understanding generator expressions
Finding the value in the middle
Finding the most popular value
Creating Python modules and applications
Creating and using a module
Creating an application module
Creating a hybrid module
Creating our own classes of objects
Using a class definition
Comparisons and correlations
Computing the standard deviation
Computing a standardized score
Comparing a sequence and an iterable
Computing a coefficient of correlation
Writing high-quality software
Building a self-testing module and a test module
Creating more sophisticated tests
Adding doctest cases to a class definition
Solving problems - analyzing some interesting datasets
Getting some more data
Further research
Summary
Index
There you have it. You can easily tell that the sub-chapters and topics are all basic python data manipulation based, as opposed to security examples. This, coupled with the fact that the code is generally not compatible with python 2.7 code, makes this book for the narrower audience of those wanting to lean general programming vs those looking to learn new security related techniques. That said, the book keeps the reader entertained with its 'secret agent' them. Don't take my word for it, check out the preview offered by O'Reilly, and dig into some python 3 web queries and data parsing.