flywheel.engine module

Query engine

class flywheel.engine.Engine(dynamo=None, namespace=(), default_conflict='update')

Bases: object

Query engine for models

Parameters:

dynamo : dynamodb3.DynamoDBConnection, optional

namespace : list or str, optional

String prefix or list of component parts of a prefix for models. All table names will be prefixed by this string or strings (joined by ‘-‘).

default_conflict : {‘update’, ‘overwrite’, ‘raise’}, optional

Default setting for delete(), save(), and sync() (default ‘update’)

Notes

The engine is used to save, sync, delete, and query DynamoDB. Here is a basic example of saving items:

item1 = MyModel()
engine.save(item1)
item1.foobar = 'baz'
item2 = MyModel()
engine.save([item1, item2], overwrite=True)

You can also use the engine to query tables:

user = engine.query(User).filter(User.id == 'abcdef).first()

# calling engine() is a shortcut for engine.query()
user = engine(User).filter(User.id == 'abcdef).first()

d_users = engine(User).filter(User.school == 'MIT',
                              User.name.beginswith_('D')).all()

# You can pass in equality constraints as keyword args
user = engine(User).filter(id='abcdef').first()

Scans are like queries, except that they don’t use an index. Scans iterate over the ENTIRE TABLE so they are REALLY SLOW. Scans have access to additional filter conditions such as “contains” and “in”.

# This is suuuuuper slow!
user = engine.scan(User).filter(id='abcdef').first()

# If you're doing an extremely large scan, you should tell it to return
# a generator
all_users = engine.scan(User).gen()

# to filter a field not specified in the model declaration:
prince = engine.scan(User).filter(User.field_('bio').beginswith_(
           'Now this is a story all about how')).first()

connect(*args, **kwargs)

Connect to a specific host

connect_to_host(**kwargs)

Connect to a specific host

connect_to_region(region, **kwargs)

Connect to an AWS region

create_schema(test=False, throughput=None)

Create the DynamoDB tables required by the registered models

Parameters:

test : bool, optional

If True, perform a dry run (default False)

throughput : dict, optional

If provided, override the throughputs of the Models when creating them. Details below.

Returns:

names : list

List of table names that were created

Examples

The throughput argument is a mapping of table names to their throughputs. The throughput is a dict with a ‘read’ and ‘write’ value. It may also include the names of global indexes that map to their own dicts with a ‘read’ and ‘write’ value.

engine.create_schema(throughput={
    'table1': {
        'read': 4,
        'write': 10,
        'gindex-1': {
            'read': 6,
            'write': 3,
        }
    }
})

default_conflict

Get the default_conflict value

Notes

The default_conflict setting configures the default behavior of save(), sync(), and delete(). Below is an explanation of the different values of default_conflict.

default_conflict	method	default
‘update’
	save	overwrite=True
	sync	raise_on_conflict=True
	delete	raise_on_conflict=False
‘overwrite’
	save	overwrite=True
	sync	raise_on_conflict=False
	delete	raise_on_conflict=False
‘raise’
	save	overwrite=False
	sync	raise_on_conflict=True
	delete	raise_on_conflict=True

delete(items, raise_on_conflict=None)

Delete items from dynamo

Parameters:

items : list or Model

List of Model objects to delete

raise_on_conflict : bool, optional

If True, raise exception if the object was changed concurrently in the database (default set by default_conflict)

Raises:

exc : dynamo3.ConditionalCheckFailedException

If overwrite is False and an item already exists in the database

Notes

Due to the structure of the AWS API, deleting with raise_on_conflict=False is much faster because the requests can be batched.

delete_key(model, pkeys=None, **kwargs)

Delete one or more items from dynamo as specified by primary keys

Parameters:

model : Model

pkeys : list, optional

List of primary key dicts

**kwargs : dict

If pkeys is None, delete only a single item and use kwargs as the primary key dict

Returns:

count : int

The number of deleted items

Notes

If the model being deleted has no range key, you may use strings instead of primary key dicts. ex:

>>> class Item(Model):
...     id = Field(hash_key=True)
...
>>> items = engine.delete_key(Item, ['abc', 'def', '123', '456'])

delete_keys(model, pkeys=None, **kwargs)

Delete one or more items from dynamo as specified by primary keys

Parameters:

model : Model

pkeys : list, optional

List of primary key dicts

**kwargs : dict

If pkeys is None, delete only a single item and use kwargs as the primary key dict

Returns:

count : int

The number of deleted items

Notes

If the model being deleted has no range key, you may use strings instead of primary key dicts. ex:

>>> class Item(Model):
...     id = Field(hash_key=True)
...
>>> items = engine.delete_key(Item, ['abc', 'def', '123', '456'])

delete_schema(test=False)

Drop the DynamoDB tables for all registered models

Parameters:

test : bool, optional

If True, perform a dry run (default False)

Returns:

names : list

List of table names that were deleted

exists(model, key_or_item, range_key=None, consistent=False)

Check if an item exists in the database

Parameters:

model : dynamodb3.Model

The model class of the item to check

key_or_item : dict or dynamodb3.Model or object

Either the value of the hash key, a model instance, or a dict that contains the primary key.

range_key : object, optional

Value of the range key (if the previous argument is the hash key)

consistent : bool, optional

Perform a consistent read from dynamo (default False)

get(model, pkeys=None, consistent=False, **kwargs)

Fetch one or more items from dynamo from the primary keys

Parameters:

model : Model

pkeys : list, optional

List of primary key dicts

consistent : bool, optional

Perform a consistent read from dynamo (default False)

**kwargs : dict

If pkeys is None, fetch only a single item and use kwargs as the primary key dict.

Returns:

items : list or object

If pkeys is a list of key dicts, this will be a list of items. If pkeys is None and **kwargs is used, this will be a single object.

Notes

If the model being fetched has no range key, you may use strings instead of primary key dicts. ex:

>>> class Item(Model):
...     id = Field(hash_key=True)
...
>>> items = engine.get(Item, ['abc', 'def', '123', '456'])

get_schema()

Get the schema for the registered models

query(model)

Create a table query for a specific model

Returns:

query : Query

refresh(items, consistent=False)

Overwrite model data with freshest from database

Parameters:

items : list or Model

Models to sync

consistent : bool, optional

If True, force a consistent read from the db. (default False)

register(*models)

Register one or more models with the engine

Registering is required for schema creation or deletion

save(items, overwrite=None)

Save models to dynamo

Parameters:

items : list or Model

overwrite : bool, optional

If False, raise exception if item already exists (default set by default_conflict)

Raises:

exc : dynamo3.ConditionalCheckFailedException

If overwrite is False and an item already exists in the database

Notes

Overwrite will replace the entire item with the new one, not just different fields. After calling save(overwrite=True) you are guaranteed that the item in the database is exactly the item you saved.

Due to the structure of the AWS API, saving with overwrite=True is much faster because the requests can be batched.

scan(model)

Create a table scan for a specific model

Returns:

scan : Scan

sync(items, raise_on_conflict=None, consistent=False, constraints=None, no_read=False)

Sync model changes back to database

This will push any updates to the database, and ensure that all of the synced items have the most up-to-date data.

Parameters:

items : list or Model

Models to sync

raise_on_conflict : bool, optional

If True, raise exception if any of the fields that are being updated were concurrently changed in the database (default set by default_conflict)

consistent : bool, optional

If True, force a consistent read from the db. This will only take effect if the sync is only performing a read. (default False)

constraints : list, optional

List of more complex constraints that must pass for the update to complete. Must be used with raise_on_conflict=True. Format is the same as query filters (e.g. Model.fieldname > 5)

no_read : bool, optional

If True, don’t perform a GET on models with no changes. (default False)

Raises:

exc : dynamo3.CheckFailed

If raise_on_conflict=True and the data in dynamo fails the contraint checks.

update_field(item, name, value=<object object>, action='PUT', constraints=None)

Update the value of a single field

Note that this method bypasses field validators and will ignore any special behavior around Composite fields.

Parameters:

item : Model

The model to update

name : str

The name of the field to update

value : object, optional

The new value for the field. Default will use the value currently on the model.

action : str, optional

PUT, ADD, or DELETE. (default PUT)

constraints : list, optional

List of constraints that must pass for the update to complete. Format is the same as query filters (e.g. Model.fieldname > 5)

update_schema(test=False, throughput=None)

Updates the DynamoDB table global indexes required by the registered models

Parameters:

test : bool, optional

If True, perform a dry run (default False)

throughput : dict, optional

If provided, override the throughputs of the Models when creating them. Details below.

Returns:

names : list

List of table names that were updated

Examples

The throughput argument is a mapping of table names to their throughputs. The throughput is a dict with a ‘read’ and ‘write’ value. It may also include the names of global indexes that map to their own dicts with a ‘read’ and ‘write’ value.

engine.create_schema(throughput={
    'table1': {
        'gindex-1': {
            'read': 6,
            'write': 3,
        }
    }
})