How To Install Graph Tool

A working surround for Geospatial assay with Docker, Python and PostgreSQL

This is a brusk version of my search for a working surround for storing, processing and visualising geospatial data. It involves a PostgreSQL server running locally, a Docker image with GDAL and with Python3 used as interpreter in PyCharm Professional, a local installation of QGIS (not necessary).

Objectives

The point of this article is to create a Docker image that can be used as a working surround. It tin be thought of as a sort of extended Python virtual environs where we install Python, its packages and also other libraries that are part of the OS.

My piece of work surroundings on a Mac OSX. On the local host, I installed PostgreSQL and QGIS with Homebrew. For GDAL, Python3 and all its packages, I use a Docker container. PyCharm links all these pieces together under one unmarried umbrella.

In this particular case, my objectives are:

• Store geospatial data from a PostgreSQL database and exist able to query with Python and QGIS;
• Employ ogr2ogr and other GDAL functionalities to be able to read, catechumen and write all sort of geospatial formats;
• Exist able to write Python lawmaking and test it live in a Python crush;
• Perform graph assay with graph-tool.

References

A practiced guide to Docker and why information technology is good practise to use it in information scientific discipline projects can be found in the excellent article beneath by Hamel Husain.

Part of my solutions are based on his tutorial and I suggest you read it beginning earlier continuing, especially if you are not familiar with Docker.

graph-tool comes equally a Python package, but is more than like a Python wrapper for a C library. It can be quite a headache to install and pretty much incommunicable to practise and so in a virtual environment.

EDIT: graph-tool is now on homebrew and this makes life easier. If you want it in a virtual environs, a way to go is to install it in the system's Pyhton (2.7 or 3.five+) and create a virtual environs by copying the arrangement packages with virtualenv --systam-site-packages.

GDAL (Geospatial Data Abstraction Library) is a translator library for raster and vector geospatial data formats. It is an essential element in the toolbox of the proficient geo-analyst. It is also notoriously painful to setup.

homebrew is a package manager for MacOS

Given these bounds, I thought I would share my experience in putting everything together. You can find the files in my Github repo.

Requirements

I assume y'all are on a Mac OSX with homebrew and docker 17.09 installed.

You should too know well-nigh Docker (you probably wouldn't exist here, otherwise…), exist able to employ Terminal and know the nuts and bolts of the Ubuntu shell and apt-get.

An unfortunate note: only the professional version of PyCharm can take remote virtual environments, therefore PyCharm Community Edition (the free one) will not work with Docker.

What we are going to do

Here are the steps in this tutorial:

• brew install PostgreSQL and QGIS (really, QGIS is not strictly necessary, it's simply a bonus)
• Create an Ubuntu 17.10 Docker image (I'll write in italic the terminology specific to the Docker universe) with Python3 installed with some useful packages and with the GDAL library (this is the step that can be customised to include any relevant Python package).
• Go an extra stride by opening an interactive Docker container based on the image above and manually installing graph-tool.
• Commit the results to the Docker paradigm.
• Setup PyCharm Professional person to use the Docker prototype as an interpreter and create a shared binder to be able to read/write data back and forth from the image.
• Connect Python with Postgresql.

You can find some of the files needed for the tutorial in this Github repository.

Information technology's a lot of stuff. Let'southward go started.

PostgreSQL installation

With homebrew, the installation of PostgreSQL is quite easy. From the system last:

          $ brew install postgresql        

Once installed, start the server with:

          $ brew services kickoff postgresql        

Check the status:

          $ brew services list        

Terminate the server:

          $ brew services stop postgresql        

Create a Python environment in Docker

The beginning step is to create a Docker image. Create a new directory from the Mac OSX Concluding:

          $ mkdir MyDockerImages        

Enter the directory:

          $ cd myDockerImages        

Create a file called Dockerfile:

          $ touch Dockerfile        

Copy and paste the post-obit content into Dockerfile (see the comments for explanations):

Have note of the creation of the directory /ds: it will be used after on, to share data between the Docker image and the host (your local estimator).

With this Dockerfile , we tin now build a Docker epitome:

          $ docker build -t my_env -f ./Dockerfile ./        

Install graph-tool

Detailed instructions to install graph-tool tin be institute here.

The lines to install graph-tool could accept been included in the Dockerfile. But since you might be asked to input some information from the bash beat out during the installation, I did not find a better fashion than installing it manually from a fustigate created by running the Docker paradigm interactively.

The key steps are:

• Add together a .listing file with the link to the server where graph-tool is hosted so that apt-get knows where to look for.
• Verify the package based on the public key provided in the installation instructions.
• Run apt-get update and install the package.

Run a container interactively with the selection -it :

          $ docker run -it my_env        

From within the container, the Ubuntu bash should look something like:

          root@539265d4463b        

Keep note of the number after @ because that'due south the container'southward ID which we will use in a few lines as CONTAINERID. From the container's bash, type:

Fustigate script for a Ubuntu Docker container to install graph-tool.

graph-tool is now installed, we just demand to commit the changes to the prototype so that they don't get lost when we close the container:

          $ docker commit -m "My note" -a "theAuthor" CONTAINERID my_env        

Connect everything with PyCharm

My workflow is a lot of trial and error and often I examination short lines of code, bank check the data or the type of some Python elements. For this, I demand a mix of code editor and interactive beat. PyCharm, too providing a way of structuring projects and folders, offers an easy way of jumping back and forth betwixt the two.

Nosotros are going to use the Docker epitome that we named "my_env" as an interpreter for a projection in PyCharm 2022 Professional so that nosotros can use all the packages and features nosotros have just installed. Here yous tin find the official documentation of the topic.

Open up PyCharm and create a new project. Notation that at this point you cannot add directly the Docker image as an interpreter considering of a current PyCharm limitation, as explained here. We need get-go to utilise a system interpreter so change it. Let'southward beginning open the project with a system interpreter and then go to PyCharm/Preferences/Project/Project Interpreter and select "Add remote".

Select the "Docker" and type in the missing data as in the prototype below.

The last step is to create a link to share the data between the host (your computer) and the Docker image. This will permit united states of america to read/salvage files such every bit csv, images, graphs, etc. From the card "PyCharm/Preferences/Build, Execution, Deployment/" go to Docker and add together a "Path mapping".

When creating a "Path mapping" you volition be asked to give a "Virtual Machine Path", which in our case is /ds (see the Dockerfile, when we created the Docker prototype), and a local path, which is the project data binder you wish to employ to store your data.

Connect Python to PostgreSQL

In lodge to connect from a Docker epitome to a server on the host, we are going to use the Mac specific DNS proper noun docker.for.mac.localhost, as explained here. In fact, it'south enough to replace 'localhost' with 'docker.for.mac.localhost' in the connectedness string in Python. Below is an example using psycopg2 to create the connection, run a elementary query and fetch the results:

PostgreSQL connection test in Python3.6.

Bonus track: QGIS installation

For QGIS, nosotros first need to install some required packages into the system Python. From the system concluding:

          $ pip install psycopg2 requests time to come jinja2 pygments        

Check package options if you want to install specific supports:

          $ mash info qgis2        

Install QGIS two.eighteen (soon 3):

          $ brew install qgis2        

Create a symlink of the awarding, so that an icon volition appear in the Application folder:

          $ ln -south /usr/local/Cellar/qgis2/ii.18.14/QGIS.app /Applications        

Disclosure

I don't work or have any contact with the PyCharm team at JetBrains, I am actually a paying user.

Please, get in impact!

I'd love to hear your comments, please go far bear upon on: Twitter, Github, Linkedin.

Source: https://towardsdatascience.com/a-working-environment-for-geospatial-analysis-with-docker-python-and-postgresql-670c2be58e0a

Posted by: burkelapet1958.blogspot.com

Share this post