Importing a fixed-width file in Python

How to import a fixed-width file in Python to a dictionary (or a data frame)

Photo by iam_os on Unsplash

The most common text format for data is the comma (or semicolumn, or pipe, etc.) seperated format. But once in a while reading a fixed width text file is required.

In this type of file each column has a fixed width (hence the name). There is no separator like in the CSV format. When the file is opened in a text editor it kind of looks like a spreadsheet with left or right aligned values. This makes it for humans easy to read, but for a computer a little bit harder.

As an example, we use historical weather information from the dutch meteorology institute KNMI (source). Originally, it is a combination of a comma separated and fixed width formats. To use it as an example, all comma’s are replaced by white spaces. The first 51 lines describe the contents of the file followed by the table of data:

Part of the weather information (source: KNMI)

Pandas has the method pandas.read_fwf(…) that can detect columns also but it is a good exercise to develop your own. When the infer method is used, the function looks at the first 100 lines of data to determine the columns. But when you are not using Pandas, or are…