I ended up finding a solution to this. Effectively, what I've done is to generate the Marshmallow schema exclusively for translation from the DynamoDB into the Python objects. All Schema classes have @post_load methods that translate into the Python objects and all fields are labeled with the type they need to be in the Python world, not the database world.

When validating the input from the REST API and ensuring that no bad data is allowed to get into the database, I call MySchema().validate(input_json), check to see that there are no errors, and if not, dump the input_json into the database.

This leaves only one extra problem which is that the input_json needs to be cleaned up for entry into the Database, which I was previously doing with Marshmallow. However, this can also easily be done by adjusting my JSON decoder to read Decimals from floats.

So in summary, my JSON decoder is doing the work of recursively walking the data structure and converting Float to Decimal separately from Marshmallow. Marshmallow is running a validate on the fields of every object, but the results are only checked for errors. The original input is then dumped into the database.

I needed to add this line to do the conversion to Decimal.

app.json_decoder = partial(flask.json.JSONDecoder, parse_float=decimal.Decimal)

My create function now looks like this. Notice how the original input_json, parsed by my updated JSON decoder, is inserted directly into the database, rather than any data mundged output from Marshmallow.

@app.route("/machine/<uuid:machine_id>", methods=['POST'])
def create_machine(machine_id):
    input_json = request.get_json() # Already ready to be DB input as is.
    errors = MachineSchema().validate(input_json)
    if errors:
      return jsonify({"status": "failure",message = dumps(errors)})
    else:
      input_json['id'] = machine_id
      dynamodb.Table('machine').put_item(Item=input_json)
      return jsonify({"status", "success", error_message = ""})

I just managed to do a complete dump and "restore" using bchew/dynamodump:

git clone [email protected]:bchew/dynamodump.git

Notice the --schemaOnly option in the documentation https://github.com/bchew/dynamodump. Command was:

./dynamodump.py -m backup --schemaOnly --region foo-region --host localhost --srcTable '*' --port 8000 --accessKey fooKey --secretKey barKey

Then you can use the -m restore mode to put the data or schema back into a local dynamodb or wherever desired :)

With that said, I still find it unbelievable how bad is the amazon dynamodb tool-chain. Come on guys.

I posted the answer to the same question here:

https://stackoverflow.com/questions/42100210/how-to-export-an-existing-dynamo-table-schema-to-json#answer-67695947

Summary:

AWS_PROFILE=xyz aws dynamodb describe-table --table-name MyTable > mytable_full.json

# Pull out just what is minimally needed for table-creation:
#
#  TableName
#  KeySchema
#  AttributeDefinitions (must only contain attributes used in keys)
#  Global/Local Secondary Indexes
#  Defaults BillingMode to PAY_PER_REQUEST
#   (any provisioning can be set up manually based on load)

jq <mytable_full.json '.Table | {TableName, KeySchema, AttributeDefinitions} + (try {LocalSecondaryIndexes: [ .LocalSecondaryIndexes[] | {IndexName, KeySchema, Projection} ]} // {}) + (try {GlobalSecondaryIndexes: [ .GlobalSecondaryIndexes[] | {IndexName, KeySchema, Projection} ]} // {}) + {BillingMode: "PAY_PER_REQUEST"}' ​>mytable.json

AWS_PROFILE=xyz aws dynamodb create-table --cli-input-json file://mytable.json