AWS DynamoDB
Table of Contents
- DynamoDB Overview
- DynamoDB API
- DynamoDB Partitions
- DynamoDB Secondary Indexes
- DynamoDB Consistency
- DynamoDB Capacity & Throughput
- DynamoDB Pricing
- DynamoDB Access and Permissions Based on IAM
- DynamoDB Limits
- DynamoDB Best Practices
- Backup & Restore DynamoDB
- Manipulate DynamoDB Items with
awscli
DynamoDB.Table.update_item
DynamoDB.Table.wait_until_exists
boto3.dynamodb.conditions
DynamoDB Overview tutorial
- Tables like
People
,Music
- Items like each
{..}
in Tables - Attributes like
PersonID
,LastName
,Artist
,Price
, and so on - Primary keys are bolded attributes like
PersonID
,(Artist, SongTitle)
- Primary key can be either a single
partition key
or a pair ofpartition key
andsort key
- Primary key attribute must be a scalar(like
string
,number
, etc) - Secondary Indexes like
GenreAlbumTitle
- Secondary Indexes can be either
Global
orLocal
Global
consists of apartition key
and asort key
that can be different from those on the tableLocal
has the samepartition key
as the table but a differentsort key
DynamoDB API reference
CreateTable
DescribeTable
ListTables
UpdateTable
DeleteTable
PutItem
BatchWriteItem (up to 25 items)
GetItem (uses primary key)
BatchGetItem (up to 100 items)
Query
Scan
UpdateItem
DeleteItem
BatchWriteItem
ListStreams
DescribeStream
GetShardIterator
GetRecords
Actual using examples are following:
"GetItem"
{
TableName: "Music",
Key: {
"Artist": "No One You Know",
"SongTitle": "Call Me Today"
}
}
"Query"
{
TableName: "Music",
KeyConditionExpression: "Artist = :a and SongTitle = :t",
ExpressionAttributeValues: {
":a": "No One You Know",
":t": "Call Me Today"
}
}
"Query using Index"
{
TableName: "Music",
IndexName: "GenreAndPriceIndex",
KeyConditionExpression: "Genre = :genre",
ExpressionAttributeValues: {
":genre": "Rock"
},
};
DynamoDB Partitions discussion
- DynamoDB calculates the hash value of
Dog
, yielding the partition in which these items are stored. - DynamoDB then scans the sort key attribute values until it finds
Fido
.
DynamoDB Secondary Indexes discussion
- This requires separated capacity management from those of its base table
GSI
- A global secondary index is considered "global" because queries on the index can span all of the data in the base table, across all partitions.
- When you create an index, you specify which attributes will be copied, or projected, from the base table to the index.
- DynamoDB maintains indexes automatically.
- In this case,
Artist
andSongTitle
are projected attributes - Global secondary indexes are composed of partitions.
Attribute Projections
- A projection is the set of attributes that is copied from a table into a secondary index.
- There are three options for attribute projections:
KEYS_ONLY
INCLUDE
ALL
LSI
- A local secondary index is "local" in the sense that every partition of a local secondary index is scoped to a base table partition that has the same partition key value.
DynamoDB Consistency discussion
- DynamoDB supports eventually consistent and strongly consistent reads.
DynamoDB Capacity & Throughput discussion
- Read unit is 4 KB per second (twice when using eventually consistent reads, not strongly consistent reads)
- Write unit is 1 KB per second
- Each unit performs one request per second(if the operation requires the size less than the unit size) . In other words, with 5 read capacity unit, you can handle 5 requests per second.
- Options for determining throughput: Auto scaling, Provisioned, Reserved
Query—reads multiple items that have the same partition key value. All of the items returned are treated as a single read operation, where DynamoDB computes the total size of all items and then rounds up to the next 4 KB boundary. For example, suppose your query returns 10 items whose combined size is 40.8 KB. DynamoDB rounds the item size for the operation to 44 KB. If a query returns 1500 items of 64 bytes each, the cumulative size is 96 KB.
DynamoDB Pricing
- Based on WCU(Write Capacity Unit), RCU(Read Capacity Unit), and Storage
DynamoDB Access and Permissions Based on IAM discussion
If you use
dynamodb:Attributes
, you must specify the names of all of the primary key and index key attributes for the table and any secondary indexes that are listed the in the policy. Otherwise, DynamoDB can't use these key attributes to perform the requested action.
DynamoDB Limits discussion
DynamoDB Best Practices discussion
Backup & Restore DynamoDB howto
Manipulate DynamoDB Items with awscli
howto awscli
#a
- an expression attribute name, mostly used with projection expressions
:a
- an expression attribute value, mostly used with update expressions
Preventing Overwrites by using condition-expression
aws dynamodb put-item \
--table-name ProductCatalog \
--item file://item.json \
--condition-expression "attribute_not_exists(Id)"
References for operators like attribute_not_exists
are here.
Attributes
- Also, the following characters have special meaning in DynamoDB:
#
(hash) and:
(colon). - Although DynamoDB allows you to use these reserved words and special characters for names, we recommend that you avoid
DynamoDB.Table.update_item
reference boto3
key = {'name': name}
expr = ','.join(['SET age = :age',
'SET sex = :sex'])
attrs = {':age': 32, ':sex': 'M'}
table.update_item(Key=key,
UpdateExpression=','.join(exprs),
ExpressionAttributeValues=attrs)
- An expression attribute value(the key of
attrs
dict) must begin with:
and be followed by one or more alphanumeric characters.
- https://boto3.readthedocs.io/en/latest/reference/services/dynamodb.html#DynamoDB.Table.update_item
- https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.UpdateExpressions.html
- https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ExpressionAttributeValues.html
DynamoDB.Table.wait_until_exists
reference boto3
table.waituntilexists() doesn't expose the waiter parameters. To manually do the operation using client:
waiter = resource.meta.client.get_waiter('table_exists')
waiter.wait(TableName=table.name, WaiterConfig={'Delay': 5})
setattr(g, attr, table)
boto3.dynamodb.conditions
reference boto3
- Use
boto3.dynamodb.conditions.Key
andboto3.dynamodb.conditions.Attr
when making API requests. - You can use
|
and&
to compose these condition objects.