Getting Your Data into Elasticsearch: Let Me Count the Ways

on YouTube ↗10m

" summary

Shows concrete ingestion patterns: single-document indexing via client.index for quick one-offs, file loaders parsing CSV/JSON into dicts, web-app hooks using Flask forms or Django signals to build objects, and a DSL that defines mappings and injects transforms with a Django-like API. Uses helpers.bulk for thousands to millions with built-in retries and chunking. Demonstrates Pandas-style import/export with Eland. Example dataset is a toy "is my cat evil" corpus and all approaches are shown working end-to-end.

AI-written summary · about

" chapters (7)

1. 0:00Start
2. 0:05Getting Your Data into Elasticsearch Jay Miller Developer Advocate, Elastic
3. 1:33Prep pip install elasticsearch
4. 1:59Simple Data elasticsearch-py client.index()
5. 4:25From Inside an Application elasticsearch-dsl
6. 6:04A lot of data replace client.index with helpers.bulk
7. 6:25A lot of data, import elasticsearch.helpers as helpers import csv from connection import client, index

" related talks

• 47mMake sense of your (big) data using ElasticsearchHonza Kral · PyData Berlin 2014
• 57mAn Introduction to Elasticsearch for .NET DevelopersSteve Gordon · dotnetdays 2023
• 41mBetter Data Exploration and Visualisation with Elasticsearch and KibanaEmanuil Tolev · Big Data Conference Europe 2019
• 37mBeyond the basics with ElasticsearchHonza Kral · EuroPython 2015
• 36mElasticSearch and Redis How and When to Use ThemTim Marcinowski · PyData DC 2016
• 01h 01mBest Practices With ElasticsearchRoy Russo · Devnexus 2020
• 38mElasticSearch in actionThijs Feryn · Codemotion Rome 2016
• 01h 04mDiving into Elasticsearch with .NETRuss Cam · NDC Sydney 2016