Python and VTK

I recently have been working on moving data gathered in vitro as the geometric basis for some computational fluid dynamics (CFD). simulations I am running. The simulations are solved using openFOAM, therefore, I import the geometry as a series of .STL files.

The idea is that the data provided to me will be able to describe where the solid–fluid boundary is. From this I should be able to generate an .STL surface. The most reliable way (I have observed) to do this, given the data I am provided, is to generate a volume such that solid and fluid phases are distinguishable. This allows a iso-surface (and from this an .STL) to be generated.

I can employ Tecplot or Paraview to do this assuming I have an appropriate data file. Rather than painstakingly duplicate the VTK data format IO for paraview I decided to use the VTK python bindings and generate the files, and later the contours, myself.

VTK is an excellent tool. The python bindings are comprehensive and despite the package size I managed to get things moving without too much trouble. The interface to NumPy arrays allows it to interface nicely with any python based calculations I had. The errors messages were informative and the Doxygen documentation has decent descriptions for many classes. All of the classes even have help available in the interpreter. This is somewhat hidden (you need to use dir to get the available functions and ask for help for each of them individually).

The downsides: Python examples are sparse compared with C++ / tcl and some of the classes have very similar functions with slightly unpredictable behaviour i.e.

# vtk_data is of type vtk.vtkImageData()

im_FFT = vtk.vtkImageFFT()

im_FFT.SetInputConnection(vtk_data.GetOutputConnection())

im_FFT.SetInput(vtk_data)

The examples (and to some extent the book) presume that you have a compatible data file to start with. There are no examples of how to bring in large quantities of data from another part of a program.

Recommendations: Get yourself a copy of the vtk book for the first few days of working with VTK. It introduces concepts in a straightforward manner and increased my understanding substantially. After you are familiar with VTK it is not required.

Next project: Tecplot (.plt) to .vt* converter.. I have a very limited version working, however, it requires work to be robust.

Binary data and Python: Just use NumPy!

To post-process some CFD data, I have manipulated binary files generated for Tecplot with Python. The challenge here is how to import large vectors of binary numbers into NumPy ndarrays while processing binary metadata.

This task appears straightforward as shown below:

import numpy as np
import struct

file_in = 'strange_binary_format.dat'
fd = open(file_in,'rb')
# buffer data -- this is bulky
buffer = fd.read()
# read 1000 doubles from the buffer from byte "position" forward
position = 0
no_of_doubles = 1000
read_format = str(no_of_doubles) + 'd'
read_size = struct.calcsize(read_format)
# put data into numpy arrays . . this is very slow and memory intensive
# might be due to struct.unpack returning as a tuple of floats
numpy_data =np.array(struct.unpack(read_format, buffer[position:
            (position + read_size)]))

However, this method is inefficient (and possibly cause memory leaks!). The struct.unpack function returns a tuple with 1000 individual floats resulting in significant overhead. The result is both memory intensive and slow. A later attempt is shown below:

import numpy as np
import struct

file_in = 'strange_binary_format.dat'
fd = open(file_in,'rb')
position = 0
no_of_doubles = 1000
# move to position in file
fd.seek(position,0)

# straight to numpy data (no buffering) 
numpy_data = np.fromfile(fd, dtype = np.dtype('d'), count = no_of_doubles)

The NumPy function fromfile is significantly more efficient in terms of  both time and memory.

From this experience I have a rule for numerically intensive computing with Python: NumPy / SciPy functions will almost always be faster!