FTS#
pyinaturalist_convert.fts
Build and search a full text search tables for taxa and observations using FTS5.
Extra dependencies: sqlalchemy
(only for building the database; not required for searches)
Taxon Autocomplete#
TaxonAutocompleter
works similarly to the API endpoint
get_taxa_autocomplete()
, which powers the taxon autocomplete
feature on inaturalist.org:
Build database with all taxa from GBIF archive:
>>> from pyinaturalist_convert import (
... aggregate_taxon_db, enable_logging, load_dwca_tables, load_fts_taxa
... )
>>> # Optional, but recommended:
>>> enable_logging()
>>> load_dwca_tables()
>>> aggregate_taxon_db()
>>> # Load FTS table for all languages (Defaults to English names only):
>>> load_fts_taxa(language='all')
Note
Running aggregate_taxon_db()
will result in more accurate search rankings based
on taxon counts, but will take a couple hours to complete.
Search taxa:
>>> from pyinaturalist_convert import TaxonAutocompleter
>>> ta = TaxonAutocompleter()
>>> # Search by scientific name
>>> ta.search('aves')
[
Taxon(id=3, name='Aves'),
Taxon(id=1043988, name='Avesicaria'),
...,
]
>>> # Or by common name
>>> ta.search('frill')
[
Taxon(id=56447, name='Acid Frillwort'),
Taxon(id=614339, name='Antilles Frillfin'),
...,
]
>>> # Or by common name in a specific language
>>> ta.search('flughund', language='german')
Observation Autocomplete#
ObservationAutocompleter
adds additional observation search features not available in
the web UI.
Query all of your own observations:
>>> from pyinaturalist import iNatClient
>>> client = iNatClient()
>>> observations = client.observations.search(user_id='my_username').all()
Create table and index observations:
>>> from pyinaturalist_convert import create_observation_fts_table, index_observation_text
>>> create_observation_fts_table()
>>> index_observation_text(observations)
Search observations:
>>> from pyinaturalist_convert import ObservationAutocompleter
>>> obs_autocompleter = ObservationAutocompleter
>>> obs_autocompleter.search('test')
[
(12345, 'test description text'),
(67890, 'test comment text'),
]
Main classes and functions:
Observation autocomplete search. |
|
Taxon autocomplete search. |
|
Create a SQLite FTS5 table for taxonomic names |
|
Create a SQLite FTS5 table for observation text |
|
Index observation text in FTS table: descriptions, places, comments, and identification comments. |
|
Create full text search tables for taxonomic names. |
- class pyinaturalist_convert.fts.ObservationAutocompleter(db_path=PosixPath('/home/docs/.local/share/pyinaturalist/observations.db'), limit=10, truncate_match_chars=50)#
Bases:
object
Observation autocomplete search. Runs full text search on observation descriptions, comments, identification comments, and place names.
- Parameters:
- search(q, fields=None)#
Search for observations by text.
- class pyinaturalist_convert.fts.TaxonAutocompleter(db_path=PosixPath('/home/docs/.local/share/pyinaturalist/observations.db'), limit=10)#
Bases:
object
Taxon autocomplete search. Runs full text search on taxon scientific and common names.
- Parameters:
- class pyinaturalist_convert.fts.TextField(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)#
Bases:
Enum
- COMMENT = 2#
- DESCRIPTION = 1#
- IDENTIFICATION = 3#
- PLACE = 4#
- pyinaturalist_convert.fts.create_observation_fts_table(db_path=PosixPath('/home/docs/.local/share/pyinaturalist/observations.db'))#
Create a SQLite FTS5 table for observation text
- pyinaturalist_convert.fts.create_taxon_fts_table(db_path=PosixPath('/home/docs/.local/share/pyinaturalist/observations.db'))#
Create a SQLite FTS5 table for taxonomic names
- pyinaturalist_convert.fts.index_observation_text(observations, db_path=PosixPath('/home/docs/.local/share/pyinaturalist/observations.db'))#
Index observation text in FTS table: descriptions, places, comments, and identification comments. Replaces any previously indexed text associated with these observations.
- Parameters:
observations (
Sequence
[Observation
]) – observations to index
- pyinaturalist_convert.fts.load_fts_taxa(csv_dir=PosixPath('/home/docs/.local/share/pyinaturalist/inaturalist-taxonomy.dwca'), db_path=PosixPath('/home/docs/.local/share/pyinaturalist/observations.db'), counts_path=PosixPath('/home/docs/.local/share/pyinaturalist/taxon_counts.parquet'), languages=('english',))#
Create full text search tables for taxonomic names. Requires SQLite FTS5 extension and the iNat taxonomy DwC-A archive.
- pyinaturalist_convert.fts.optimize_fts_table(table, db_path=PosixPath('/home/docs/.local/share/pyinaturalist/observations.db'))#
Some final cleanup after loading a text search table