Tools

Elastic Git provides various command line tools via the elasticgit.tools module.

$ python -m elasticgit.tools --help

Elasticgit command line tools.

usage: python -m elasticgit.tools [-h]
                                  {dump-schema,load-schema,migrate-gitmodel-repo,shell,version,resync}
                                  ...
Sub-commands:
dump-schema

Dump model information as an Avro schema.

usage: python -m elasticgit.tools dump-schema [-h] class_path
Positional arguments:
class_path python path to Class.
load-schema

Dump an Avro schema as an Elasticgit model.

usage: python -m elasticgit.tools load-schema [-h] [-m key=FieldType]
                                              [-r OldModelName=NewShiny]
                                              schema_file [schema_file ...]
Positional arguments:
schema_files path to Avro schema file.
Options:
-m, --map-field
 Manually map specific field names to Field classes. Formatted as ``field=IntegerField``
-r, --rename-model
 Manually rename a model.Formatted as ``OldModelName=NewShiny``
migrate-gitmodel-repo

Migrate a GitModel based repository layout to anElastic-Git repository layout

usage: python -m elasticgit.tools migrate-gitmodel-repo [-h]
                                                        working_dir
                                                        module_name
Positional arguments:
working_dir The directory of git model repository to migrate.
module_name The module to put the migrated data in.
shell

Load a repo and make an EG workspace available for debugging

usage: python -m elasticgit.tools shell [-h] [-m MODELS] [-n] workdir
Positional arguments:
workdir Path to the repository’s working directory.
Options:
-m, --models The models module to load.
-n=True, --no-introspect-models=True
 Do not find & load models automatically.
version

Tools for versioning & version checking a content repository

usage: python -m elasticgit.tools version [-h] -n NAME -l LICENSE -a AUTHOR
                                          [-au AUTHOR_URL] [-f FILE_NAME]
Options:
-n, --name The name to give this repository
-l, --license The license the publish this content under.
-a, --author The author
-au, --author-url
 The url where to find more information about the author
-f=.unicore.json, --file=.unicore.json
 The file to write to. Set to `-` for stdout.
resync

Tools for resyncing data in a git repository with what is in the search index.

usage: python -m elasticgit.tools resync [-h] [-c CONFIG_FILE] -m MODEL_CLASS
                                         [-s SECTION_NAME] [-i INDEX_PREFIX]
                                         [-u ES_HOST] [-p GIT_PATH]
                                         [-f MAPPING_FILE] [-r RECREATE_INDEX]
Options:
-c, --config Python paste config file.
-m, --model The model class to load.
-s=app:cmsfrontend, --section-name=app:cmsfrontend
 The section from where to read the config keys.
-i, --index-prefix
 The index prefix to use
-u, --es-host The elasticsearch url to use
-p, --git-path The path to the repository.
-f, --mapping-file
 The path to a custom mapping file.
-r=False, --recreate-index=False
 Whether or not to recreate the index from scratch.

Avro

class elasticgit.commands.avro.FieldMapType(mapping)[source]

A custom type for providing mappings on the command line for the SchemaLoader tool.

Parameters:mapping (str) – A mapping of a key to a field type
>>> from elasticgit.commands.avro import FieldMapType
>>> mt = FieldMapType('uuid=elasticgit.models.UUIDField')
>>> mt.key
'uuid'
>>> mt.field_class
<class 'elasticgit.models.UUIDField'>
>>>
class elasticgit.commands.avro.RenameType(mapping)[source]

A custom type for renaming things.

Parameters:mapping (str) – A mapping of an old name to a new name
>>> from elasticgit.commands.avro import RenameType
>>> rt = RenameType('OldName=NewName')
>>> rt.old
'OldName'
>>> rt.new
'NewName'
>>>
class elasticgit.commands.avro.SchemaDumper[source]

Dump an Avro JSON schema for an Elasticgit Model.

python -m elasticgit.tools dump-schema elasticgit.tests.base.TestPerson
dump_schema(model_class)[source]

Return the JSON schema for an elasticgit.models.Model.

Parameters:model_class (elasticgit.models.Model) –
Returns:str
get_field_info(name, field)[source]

Return the Avro field object for an elasticgit.models.Model field.

Parameters:
Returns:

dict

run(class_path)[source]

Introspect the given class path and print the schema to self.stdout

Parameters:class_path (str) – The path to the model file to introspect
class elasticgit.commands.avro.SchemaLoader[source]

Load an Avro JSON schema and generate Elasticgit Model python code.

python -m elasticgit.tools load-schema avro.json
generate_model(schema, field_mapping={}, model_renames={}, include_header=True)[source]

Generate Python code for the given Avro schema

Parameters:
  • schema (dict) – The Avro schema
  • field_mapping (dict) – An optional mapping of keys to field types that can be used to override the default mapping.
  • model_renames (dict) – An optional mapping of model names that can be used to rename a model.
Parak bool include_header:
 

Whether or not to generate the header in the source code, this is useful of you’re generating a list of model schema but don’t want the header and import statements printed every time.

Returns:

str

generate_models(schemas, field_mapping={}, model_renames={})[source]

Generate Python code for the given Avro schemas

Parameters:
  • schemas (list) – A list of Avro schema’s
  • field_mapping (dict) – An optional mapping of keys to field types that can be used to override the default mapping.
Returns:

str

run(schema_files, field_mappings=None, model_renames=None)[source]

Inspect an Avro schema file and write the generated Python code to self.stdout

Parameters:
  • schema_files (list) – The list of file pointers to load.
  • field_mappings (list) – A list of FieldMapType types that allow overriding of field mappings.
  • model_renames (list) – A list of RenameType types that allow renaming of model names
elasticgit.commands.avro.deserialize(data, field_mapping={}, module_name=None)[source]

Deserialize an Avro schema and define it within a module (if specified)

Parameters:
  • data (dict) – The Avro schema
  • field_mapping (dict) – Optional mapping to override the default mapping.
  • module_name (str) – The name of the module to put this in. This module is dynamically generated with imp.new_module() and only available during code generation for setting the class’ __module__.
Returns:

elasticgit.models.Model

>>> from elasticgit.commands.avro import deserialize
>>> schema = {
... 'name': 'Foo',
... 'type': 'record',
... 'fields': [{
...         'name': 'some_field',
...         'type': 'int',
...     }]
... }
>>> deserialize(schema)
<class 'Foo'>
>>>
elasticgit.commands.avro.serialize(model_class)[source]

Serialize a elasticgit.models.Model to an Avro JSON schema

Parameters:model_class (elasticgit.models.Model) –
Returns:str
>>> from elasticgit.commands.avro import serialize
>>> from elasticgit.tests.base import TestPerson
>>> json_data = serialize(TestPerson)
>>> import json
>>> schema = json.loads(json_data)
>>> sorted(schema.keys())
[u'fields', u'name', u'namespace', u'type']
>>>

Git Model