pyinaturalist-convert#

This package provides tools to convert iNaturalist observation data to and from a wide variety of useful formats. This is mainly intended for use with the iNaturalist API via pyinaturalist, but also works with other data sources.

Complete project documentation can be found at pyinaturalist-convert.readthedocs.io.

Formats#

Import#

CSV (From either API results or the iNaturalist export tool)
JSON (from API results)
pyinaturalist.Observation objects
Dataframes, Feather, Parquet, and anything else supported by pandas
iNaturalist GBIF Archive
iNaturalist Taxonomy Archive
iNaturalist Open Data on Amazon
Note: see API Recommended Practices for details on which data sources are best suited to different use cases

Export#

CSV, Excel, and anything else supported by tablib
Dataframes, Feather, Parquet, and anything else supported by pandas
Darwin Core
GeoJSON
GPX
SQLite
SQLite + FTS5 text search for taxonomy

Installation#

Install with pip:

pip install pyinaturalist-convert

Or with conda:

conda install -c conda-forge pyinaturalist-convert

To keep things modular, many format-specific dependencies are not installed by default, so you may need to install some more packages depending on which features you want. Each module’s docs lists any extra dependencies needed, and a full list can be found in pyproject.toml.

For getting started, it’s recommended to install all optional dependencies:

pip install pyinaturalist-convert[all]

Usage#

Export#

Get your own observations and save to CSV:

from pyinaturalist import get_observations
from pyinaturalist_convert import *

observations = get_observations(user_id='my_username')
to_csv(observations, 'my_observations.csv')

Or any other supported format:

to_dwc(observations, 'my_observations.dwc')
to_excel(observations, 'my_observations.xlsx')
to_feather(observations, 'my_observations.feather')
to_geojson(observations, 'my_observations.geojson')
to_gpx(observations, 'my_observations.gpx')
to_hdf(observations, 'my_observations.hdf')
to_json(observations, 'my_observations.json')
to_parquet(observations, 'my_observations.parquet')
df = to_dataframe(observations)

Import#

Most file formats can be loaded via pyinaturalist_convert.read():

observations = read('my_observations.csv')
observations = read('my_observations.xlsx')
observations = read('my_observations.feather')
observations = read('my_observations.hdf')
observations = read('my_observations.json')
observations = read('my_observations.parquet')

Download#

Download the complete research-grade observations dataset:

download_dwca_observations()

And load it into a SQLite database:

load_dwca_observations()

And do the same with the complete taxonomy dataset:

download_dwca_taxa()
load_dwca_taxa()

Load taxonomy data into a full text search database:

load_taxon_fts_table(languages=['english', 'german'])

And get lightning-fast autocomplete results from it:

ta = TaxonAutocompleter()
ta.search('aves')
ta.search('flughund', language='german')

Feedback#

If you have any problems, suggestions, or questions about pyinaturalist-convert, you are welcome to create an issue or discussion. Also, PRs are welcome!

API Reference#

Modules#

`pyinaturalist_convert.converters`	Base utilities for converting observation data to common formats.
`pyinaturalist_convert.csv`	Import CSV observation data from the iNaturalist bulk export tool.
`pyinaturalist_convert.db`	Data models and utilities for storing observations and related data in a relational database, including SQLite, PostgreSQL, and any other database supported by SQLAlchemy.
`pyinaturalist_convert.dwc`	Convert observations to and from Darwin Core.
`pyinaturalist_convert.dwca`	Download and convert the iNaturalist GBIF and taxonomy datasets from DwC-A to SQLite.
`pyinaturalist_convert.fts`	Build and search a full text search tables for taxa and observations using FTS5.
`pyinaturalist_convert.geojson`	Convert observations to GeoJSON FeatureCollections.
`pyinaturalist_convert.gpx`	Convert observations to GPX tracks or waypoints.
`pyinaturalist_convert.odp`	Download and convert observation, taxon, photo, and user metadata from inaturalist-open-data.

Additional utility modules#

`pyinaturalist_convert.download`	Helper utilities for downloading and extracting files, with progress bars
`pyinaturalist_convert.sqlite`	Helper utilities to load data directly from CSV into a SQLite database
`pyinaturalist_convert.taxonomy`	Helper utilities for navigating tabular taxonomy data as a tree and adding additional derived information to it.