A database worth downloading for time series analysis

Obtaining ground-truth data for geospatial analysis can be a daunting task for geospatial developers or scientists in general. This is often due to the limited manpower, as well as the limited organization, and limited budget to carry out such large tasks as a spatial census of what is happening on the Earth’s surface at a temporal resolution (the time taken between each survey) sufficient to perform time series analysis.

Today I would like to make a parenthesis and leave aside all this discouraging panorama, to present you some geospatial data that seem to me an exception to the situation, and that could be of really interesting for further time series analyses in agriculture.

This dataset are the LPIS Data (Land Parcel Identification System), for the federal state of Brandenburg (Germany), for years 2010 until 2022 (and the years to come). It is outstanding since it contains information of parcels dedicated to agriculture. Some of the information we can derived from it are the crop types, the parcel id (without any personal information of the parcel’s owner), land use code (agriculture, grassland, etc.), and year of cultivation. Although not all parcels in this location are represented, we can assure that a great amount of agricultural parcels can be depicted in this dataset for every year.

With that being said, we will go through to the steps to download the data, and after we will make it available in a Google Earth Engine account.

How to download the “raw” data

When entering Invekos Brandenburg, we can pin point the result that will lead us to the webpage of the Geoportal with geospatial datasets for the state of Brandenburg. Click on the entry “Daten aus dem Agrarförderantrag”. This is the LPIS Data for Germany. The name Invekos stands for LPIS in Germany.

01. Google Search

Once in this geoportal, the web page should look like this.

02. Geoportal Brandenburg

Now scroll down and click on the geobrocker link as shown in the yellow highlighted text.

03. Link to geobrocker

Once on the geobrocker page, click on the button “Weiter zu Bestellvorgang”.

04. Weiter zum Bestellvorgang

On the following page, you can see a small interactive map of the federal state of Brandenburg in Germany. Below you will find the button “Datenauftrag Erzeugen”, click there.

05. Button Datenauftrag erzeugen

Now you will see a page with some fields to which you can change the input parameters. For example the example below shows the parameters for downloading agricultural parcel data in Shape format, year 2022 and WGS 84 coordinate system. Then click on the “Weiter” (means “next” in german), button.

06. Settings for downloading area

Once you see the button “Warenkorb”, which is “cart” in german, you will be redirected to the status your requested data.

07. Cart

The status of your shopping cart will be displayed, and you will then be able to check out by pressing the “zu Kasse gehen” button.

08. Check out

For me as you can see below, the problem is now that I did not do a log in before making the data request. So immediately when you click on the check out, you will be redirected to the log in page. If you have credentials (user name and password), just enter them. But in this example, we are going to make a request for our username and password. So, click on the “Registrieren” button, so we can start the registering process.

09. Log in

Registration process

Below, you will find an example of the fields as I filled them out. Fill in the necessary fields to become a registered user for this web portal.

10. Registering

Accept the terms of use and click on “Eingabe überprüfen”.

11. Accepting the terms of use

Check that the information provided is correct, and click on register, to begin the registration process.

12. Checking the user information

If everthing goes well, the message “Ihre Registrierung verlief erfolgreich. Sie bekommen Ihr Passwort per Mail zugesendet.”, which means your registration has been successful, you will receive a password by email.

13. Registration confirmation

Log in

Now click on “weiter zur Anmeldung”.

14. Next steps to log in

After a few minutes, you will receive an email with your username and password.

15. User information received per E-Mail

Enter your password and username to enter the geoportal as a registered user. I had to insist sometimes, trying both: copying the information in my clipboard form my E-Mail, or entering my username and password through my keyboard. I persisted until it worked, the same may happen to you. Don’t get discouraged, and keep at it until it works. There might be a delayed until your confirmation is down in your E-Mail, and when you actually can use it to log in.

16. Log in

Download data

You can go back to your cart overview, and see the data that is ready to be requested. Click on “Kaufen” again.

17. Click on "Kaufen" (Buy) again now, as a registered user

Now your job is to wait until you find a message in your E-Mail box, about the confirmation that your data is downloadable. First you will received a message about the request, and after a message when your data is ready to be downloaded.

18. Requested data conirmation

19. Download data confirmation

And that’s basically it. By clicking on the download link found in the confirmation message in your E-Mail box, you will be redirected to a page like the one below. By clicking on the highlighted button below, the download process will start.

20. Download link available

Creating an asset in Google Earth Engine

Now that we have downloaded our data in Shape format, let’s go to our Google Earth Engine account and import the dataset for the year 2022 and add a description of the dataset that will help us to highlight its main features. irst of all let’s open the google earth engine home page, and click on the tab code editor to open our google earth engine staff in case we have already created one. For more information about how to open an account for google earth engine visit the following link: https://developers.google.com/earth-engine/guides/access

21. Opening Google Earth Engine Code Editor

On the left hand side of the code editor we can see several tabs among which is the Assets tab. Let’s click on this one.

22. Code editor Interface

When we open it we can see some of the assets we have already uploaded, if we have done so in the past. However, for the sake of this tutorial, we are going to assume that you are new to uploading assets in google earth engine.

23. Asset's tab

Now please click on the new tab to start the process of creating a new asset.

24. Creating new asset

You will see that you have several formats supported by google earth engine to be uploaded among which are CSV files, images, image collections, folders and shapefiles. This last one is the one we are interested in since the formats we downloaded in the previous section come in this type of format.

25. Uploading Shapefile

Under the header source files, you will see a button with the label select, click on it to browse the files that are part of the shapefile and that you will need to upload to create this asset.

26. Selecting Shapefile from local folder

As you can see I have selected the .shp, .shx, .dbf, and .prj extensions which are the minimum necessary to upload as far as the shapefile is concerned (I did not upload the .cpg file at the time of writing this blog post, as in the past this type of extension was not allowed, it seems it is now. I may have made a mistake and this type of extension was always allowed, but as far as I remember it was the restriction I was aware of with this extension).

27. Selecting supported files to create asset

Now press the upload button, to start the process of uploading the file to your account, as a google earth engine asset.

28. Pressing upload to trigger the task

You will see the “Task” tab in yellow, which means that the upload process is in progress.

29. Task started

By clicking on this tab you will see the progress of the upload process of this file.

30. Upload progress

At the end of the upload process, you will see the file listed in your assets.

31. Assets uploaded

Now let’s click on this dataset to see what google earth engine has already put at our disposal to describe this asset. We can observe three tabs: description, features and properties. In description we can see that no text describing the dataset has been given.

32. Description tab

In features, we can see that google earth engine already provides us with the name of the different columns in the attribute table of this asset, as well as the type of data that each column holds.

34. Features tab

When clicking on the properties tab, we can see that is also empty.

33. Properties tab

Now I wanted to add information related to this dataset, taking advantage of the Description tab. In this one we can format the text using the markdown syntax, since it is supported by this feature of the asset’s manager of google earth engine. To do this just change the “Edit” option at the top right and start adding the text that best describes the asset you just uploaded. Also at the bottom left you can find the option to add the time period this dataset covers.

34. Adding asset's description

When you again disable the edit option, you will be able to see how the description will be rendered for users to view.

35. Product’s description in viewers mode

And that’s all, you need to upload an asset to your google earth engine account. In this post, you have learned how to download LPIS data for the state of brandenburg, and how to upload it from your locally saved shape format on your computer to your google earth engine account as an asset, as well as add a description that can be rendered in a more apealign way using markdown syntax.