Update: There is a library now: https://pypi.org/project/dynamodb-json/


Here is an improved version of indiangolfer's answer. While @indiangolfer's solution works for the question, this improved version might be more useful for others who stumble upon this thread.

def unmarshal_dynamodb_json(node):
    data = dict({})
    data['M'] = node
    return _unmarshal_value(data)


def _unmarshal_value(node):
    if type(node) is not dict:
        return node

    for key, value in node.items():
        # S โ€“ String - return string
        # N โ€“ Number - return int or float (if includes '.')
        # B โ€“ Binary - not handled
        # BOOL โ€“ Boolean - return Bool
        # NULL โ€“ Null - return None
        # M โ€“ Map - return a dict
        # L โ€“ List - return a list
        # SS โ€“ String Set - not handled
        # NN โ€“ Number Set - not handled
        # BB โ€“ Binary Set - not handled
        key = key.lower()
        if key == 'bool':
            return value
        if key == 'null':
            return None
        if key == 's':
            return value
        if key == 'n':
            if '.' in str(value):
                return float(value)
            return int(value)
        if key in ['m', 'l']:
            if key == 'm':
                data = {}
                for key1, value1 in value.items():
                    if key1.lower() == 'l':
                        data = [_unmarshal_value(n) for n in value1]
                    else:
                        if type(value1) is not dict:
                            return _unmarshal_value(value)
                        data[key1] = _unmarshal_value(value1)
                return data
            data = []
            for item in value:
                data.append(_unmarshal_value(item))
            return data

It is improved in the following ways:

  • handles more data types, including lists, which were not handled correctly previously

  • handles lowercase and uppercase keys

Edit: fix recursive object bug

Answer from vekerdyb on Stack Overflow
๐ŸŒ
PyPI
pypi.org โ€บ project โ€บ dynamodb-json
dynamodb-json
JavaScript is disabled in your browser. Please enable JavaScript to proceed ยท A required part of this site couldnโ€™t load. This may be due to a browser extension, network issues, or browser settings. Please check your connection, disable any ad blockers, or try using a different browser
๐ŸŒ
Medium
medium.com โ€บ @NotSoCoolCoder โ€บ handling-json-data-for-dynamodb-using-python-6bbd19ee884e
Handling JSON data for DynamoDB using Python | by NotSoCoolCoder | Medium
July 23, 2022 - We have two ways to add these json data to DynamoDB in required format. One is a bit lengthy way using low level client as part for boto3. With this, you need to explicitly specify datatype with values.
Discussions

Python Lambda Function Parsing DynamoDB's JSON Format - Stack Overflow
Python Lambda function that gets invoked for a dynamodb stream has JSON that has DynamoDB format (contains the data types in JSON). I would like to covert DynamoDB JSON to standard JSON. PHP and ... More on stackoverflow.com
๐ŸŒ stackoverflow.com
Python Lambda: How to convert DynamoDB json from stream into regular json?
This library should help. https://pypi.org/project/dynamodb-json/ More on reddit.com
๐ŸŒ r/aws
2
5
September 26, 2020
python 3.x - How to convert simple JSON to DynamoDB JSON? - Stack Overflow
I have a simple JSON and want to convert it to DynamoDB JSON. Is there any easy way to do that? More on stackoverflow.com
๐ŸŒ stackoverflow.com
How to convert a DynamoDB Json in a regular JSON in Python? - Stack Overflow
Might help: convert single quote json data file to double quote json data file (without mangling inner quotes) ยท GitHub ... @JohnRotenstein this didn't work for me, but I found this one (github.com/Alonreznik/dynamodb-json) and it helped me a lot. More on stackoverflow.com
๐ŸŒ stackoverflow.com
๐ŸŒ
AWS re:Post
repost.aws โ€บ articles โ€บ ARRJPOEgrUSIuLebbcoETaAg โ€บ how-to-use-the-python-json-module-when-decoding-dynamodb-items
How to use the Python json module when decoding DynamoDB items | AWS re:Post
September 16, 2023 - import boto3 import json table ...m(Key={'id':'UniqueKeyId'}).get('Item') Very simple - the code writes a single item (or record) to DynamoDB and then reads the item back....
๐ŸŒ
Medium
medium.com โ€บ @maheshbhatm โ€บ converting-a-dynamodb-json-column-to-json-using-python-18e0632f4104
Converting a DynamoDB JSON Column to JSON Using Python | by Mahesh Bhat M | Medium
June 11, 2024 - Weโ€™ve explored how to convert a column from DynamoDB JSON to regular JSON using Python. By leveraging the TypeDeserializer and recursive traversal, we can efficiently handle DynamoDBโ€™s JSON format and obtain a more standard JSON representation.
๐ŸŒ
GitHub
github.com โ€บ Alonreznik โ€บ dynamodb-json
GitHub - Alonreznik/dynamodb-json: DynamoDB JSON util to load and dump strings of Dynamodb JSON format to python object and vise-versa
DynamoDB json util to load and dump strings of Dynamodb json format to python object and vise-versa
Starred by 169 users
Forked by 39 users
Languages ย  Python 100.0% | Python 100.0%
๐ŸŒ
ITNEXT
itnext.io โ€บ serialise-aws-dynamodb-json-into-python-dict-with-cerealbox-63c62bd5ba1a
Serialise AWS DynamoDB JSON with Python + CerealBox | ITNEXT
September 1, 2022 - A quick walkthrough and reference code for simplifying DynamoDB JSON serialisation process for AWS application using CerealBox & Boto3 library in Python.
๐ŸŒ
AWS
docs.aws.amazon.com โ€บ amazon dynamodb โ€บ developer guide โ€บ programming with dynamodb and the aws sdks โ€บ programming amazon dynamodb with python and boto3
Programming Amazon DynamoDB with Python and Boto3 - Amazon DynamoDB
Hereโ€™s an example of inserting an item using the client interface. Notice how all values are passed as a map with the key indicating their type ('S' for string, 'N' for number) and their value as a string. This is known as DynamoDB JSON format.
Find elsewhere
๐ŸŒ
GitHub
github.com โ€บ Alonreznik โ€บ dynamodb-json โ€บ blob โ€บ master โ€บ dynamodb_json โ€บ json_util.py
dynamodb-json/dynamodb_json/json_util.py at master ยท Alonreznik/dynamodb-json
January 31, 2019 - DynamoDB JSON util to load and dump strings of Dynamodb JSON format to python object and vise-versa - Alonreznik/dynamodb-json
Author ย  Alonreznik
Top answer
1 of 6
20

Update: There is a library now: https://pypi.org/project/dynamodb-json/


Here is an improved version of indiangolfer's answer. While @indiangolfer's solution works for the question, this improved version might be more useful for others who stumble upon this thread.

def unmarshal_dynamodb_json(node):
    data = dict({})
    data['M'] = node
    return _unmarshal_value(data)


def _unmarshal_value(node):
    if type(node) is not dict:
        return node

    for key, value in node.items():
        # S โ€“ String - return string
        # N โ€“ Number - return int or float (if includes '.')
        # B โ€“ Binary - not handled
        # BOOL โ€“ Boolean - return Bool
        # NULL โ€“ Null - return None
        # M โ€“ Map - return a dict
        # L โ€“ List - return a list
        # SS โ€“ String Set - not handled
        # NN โ€“ Number Set - not handled
        # BB โ€“ Binary Set - not handled
        key = key.lower()
        if key == 'bool':
            return value
        if key == 'null':
            return None
        if key == 's':
            return value
        if key == 'n':
            if '.' in str(value):
                return float(value)
            return int(value)
        if key in ['m', 'l']:
            if key == 'm':
                data = {}
                for key1, value1 in value.items():
                    if key1.lower() == 'l':
                        data = [_unmarshal_value(n) for n in value1]
                    else:
                        if type(value1) is not dict:
                            return _unmarshal_value(value)
                        data[key1] = _unmarshal_value(value1)
                return data
            data = []
            for item in value:
                data.append(_unmarshal_value(item))
            return data

It is improved in the following ways:

  • handles more data types, including lists, which were not handled correctly previously

  • handles lowercase and uppercase keys

Edit: fix recursive object bug

2 of 6
11

I couldn't find anything out in the wild. So, I decided to port the PHP implementation of dynamodb json to standard json that was published here. I tested this in a python lambda function processing DynamoDB stream. If there is a better way to do this, please let me know.

(PS: This is not a complete port of PHP Marshaler)

The JSON in the question gets transformed to:

{  
   "feas":{  
      "fea":[  
         {  
            "pre":"1",
            "Mo":"1",
            "Ti":"20160618184156529",
            "Fa":"0",
            "Li":"1",
            "Fr":"4088682"
         }
      ]
   }
}

def unmarshalJson(node):
    data = {}
    data["M"] = node
    return unmarshalValue(data, True)


def unmarshalValue(node, mapAsObject):
    for key, value in node.items():
        if(key == "S" or key == "N"):
            return value
        if(key == "M" or key == "L"):
            if(key == "M"):
                if(mapAsObject):
                    data = {}
                    for key1, value1 in value.items():
                        data[key1] = unmarshalValue(value1, mapAsObject)
                    return data
            data = []
            for item in value:
                data.append(unmarshalValue(item, mapAsObject))
            return data
๐ŸŒ
GitHub
github.com โ€บ Alonreznik โ€บ dynamodb-json โ€บ blob โ€บ master โ€บ setup.py
dynamodb-json/setup.py at master ยท Alonreznik/dynamodb-json
DynamoDB JSON util to load and dump strings of Dynamodb JSON format to python object and vise-versa - Alonreznik/dynamodb-json
Author ย  Alonreznik
Top answer
1 of 2
5

If you mean JsonString to Dynamodb Map, you can use boto3. Here is the example.

import boto3
import json

json_string = '{"key1": 1, "key2": "value"}'
json_obj = json.loads(json_string)

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('test-table')

table.put_item(Item={'pk': 'pk-value', 'map': json_obj})

If you just want to update the while Map attribute, you can use just JSON format the same as put_item.

json_string = '{"key1": 2, "key2": "value2"}'
json_obj = json.loads(json_string2)
rsp = table.update_item(
    Key={'pk': 'pk-value'},
    AttributeUpdates={'map': {'Value': json_obj2, 'Action': 'PUT'}}
)

However, If you want to update only specific nested attribute, you need to use UpdateExpression. For example, the below is code to update only key1 attribute to 'value3'.

nested_json_string = '{"nested": "key3"}'
nested_json_obj = json.loads(nested_json_string)

rsp = table.update_item(
    Key={'pk': 'pk-value'},
    UpdateExpression='SET #map.#key1 = :val3',
    ExpressionAttributeNames={'#map':  'map', '#key1': 'key1'},
    ExpressionAttributeValues={':val3': nested_json_obj}
)
2 of 2
1

I know this is an old question, but I came across it and the accepted answer didn't help me (it seems to suggest that you can feed boto3 with plain JSON, but it didn't work for me) and the library mentioned in the comments didn't help me either.

What did work for me was using the serializer/deserializer from boto3.dynamodb.types, basically as suggested by this answer on a very similar topic.

from boto3.dynamodb.types import TypeSerializer, TypeDeserializer
import json

serializer = TypeSerializer()
deserializer = TypeDeserializer()


# for building a DynamoDB JSON from a Python object
def serialize(item):
    serialized_item = serializer.serialize(vars(item) if hasattr(item, '__dict__') else item)

    return item if 'M' not in serialized_item else serialized_item['M']


# for building a plain JSON from a DynamoDB JSON
def deserialize(dynamodb_json_string):
    return deserializer.deserialize({'M': dynamodb_json_string})


class MyItem:
    def __init__(self, some_string_value=None, some_numeric_value=None):
        self.my_key = some_string_value
        self.my_other_key = some_numeric_value

    def __str__(self):
        return json.dumps(self, default=lambda x: x.__dict__)


if __name__ == '__main__':
    my_classy_item = MyItem("my_string_value", 5)
    my_string_item = json.loads('{"my_key": "my_string_value", "my_other_key" : 5}')

    print(serialize(my_classy_item))
    print(serialize(my_string_item))
๐ŸŒ
GeeksforGeeks
geeksforgeeks.org โ€บ python โ€บ how-to-upload-json-file-to-amazon-dynamodb-using-python
How to Upload JSON File to Amazon DynamoDB using Python? - GeeksforGeeks
July 23, 2025 - The boto3 library is a Python library that provides an interface to Amazon Web Services (AWS) services, including Amazon DynamoDB. The put_item() method on the DynamoDB client is used to insert an item into a table.
Top answer
1 of 2
2

I authored a library called cerealbox that makes it easier to perform this common conversion as follows.

from cerealbox.dynamo import from_dynamodb_json

# convert the DynamoDB image to a regular python dictionary
result = from_dynamodb_json(payload_stack['Records'][0]['dynamodb']['NewImage'])

# access the body as a regular dictionary
body = result['txt_vlr_parm_requ']['body'] 

The documentation covers how to perfom the inverse using as_dynamodb_json.

This can also be done using boto3's TypeDeserializer/TypeSerializer - a good example of this can be found here

2 of 2
0

I was able to develop a little code to convert this DynamoDB json into a regular json. I used the dynamodb_json import.

##that one in the question
payload_stack = {...} 

convert_regular_json = json.loads(payload_stack)
print(convert_regular_json)

The output:

{
    'Records': [{
        'eventID': '123456',
        'eventName': 'INSERT',
        'eventVersion': '1.1',
        'eventSource': 'aws:dynamodb',
        'awsRegion': 'sa-east-1',
        'dynamodb': {
            'ApproximateCreationDateTime': 1644956685.0,
            'Keys': {
                'body_field': 1931
            },
            'NewImage': {
                'body_field': 1931,
                'txt_vlr_parm_requ': {
                    'headers': {
                        'Authorization': 'token',
                        'correlationID': '987654321'
                    },
                    'requestContext': {
                        'requestId': '123'
                    },
                    'body': {
                        'avro_schema': '{"type":"record","namespace":"Tutorialspoint","name":"Employee","fields":[{"name":"Name","type":"string"},{"name":"Age","type":"int"}, {"name":"Address","type":"string"}, {"name":"Role","type":"string"} ]}',
                        'cluster': 'events',
                        'sigla': 'ft7',
                        'subject': 'teste-dynamo',
                        'branch': 'development',
                        'id_requisicao': 1818
                    }
                },
                'nom_tabe': 'tabela_teste',
                'cod_situ_psst_ingo': 'NOVO',
                'historic': '{"historico": [{"data/hora": "09-02-22 18:18:41", "status": "NOVO"}]}',
                'nom_arqu_bckt': 'arquivo.avro'
            },
            'SequenceNumber': '87226300000000005691898607',
            'SizeBytes': 1672,
            'StreamViewType': 'NEW_IMAGE'
        },
        'eventSourceARN': 'arn:aws'
    }]
}

And to catch the 'body' field:

catch_body_payload = var['Records'][0].get('dynamodb').get('NewImage').get('txt_vlr_parm_requ').get('body')
Top answer
1 of 2
5

I solved my problem. You can set null as follows.

from __future__ import print_function
import boto3
import codecs
import json

dynamodb = boto3.resource('dynamodb', region_name='ap-northeast-1', endpoint_url="http://localhost:8000")

table = dynamodb.Table('questionListTable')

with open("questionList.json", "r", encoding='utf-8_sig') as json_file:
    items = json.load(json_file)
    for item in items:
        q_id = item['q_id']
        q_body = item['q_body']
        q_answer = item['q_answer']
        image_url = item['image_url'] if item['image_url'] else None
        keywords = item['keywords'] if item['keywords'] else None

    print("Adding detail:", q_id, q_body)

    table.put_item(
        Item={
            'q_id': q_id,
            'q_body': q_body,
            'q_answer': q_answer,
            'image_url': image_url,
            'keywords': keywords,
        }
    )

In order to check the situation of Dynamodb, use the offline plugin of the serverless framework to run the API Gateway in the local environment. When I actually called the API using Postman, Null was properly inserted in the value.

{
  "q_id" : "001",
  "q_body" : "Where is the capital of the United States?",
  "q_answer" : "Washington, D.C.",
  "image_url" : "/Washington.jpg",
  "keywords" : [
    "UnitedStates",
    "Washington"
  ]
},
{
  "q_id" : "002",
  "q_body" : "Where is the capital city of the UK?",
  "q_answer" : "London",
  "image_url" : "null",
  "keywords" : [
    "UK",
    "London"
  ]
},
2 of 2
1

@uhiyama The following line of code in your solution can be summarized/simplified using the get() method:

image_url = item['image_url'] if item['image_url'] else None
image_url = item.get("image_url")
๐ŸŒ
AWS
aws.amazon.com โ€บ blogs โ€บ database โ€บ exploring-amazon-dynamodb-sdk-clients
Exploring Amazon DynamoDB SDK clients | Amazon Web Services
January 10, 2024 - It takes in the DynamoDB-JSON response ... from DynamoDB operations, and transforms the attribute values into their corresponding JSON data types. This allows you to work with the data in a more familiar and convenient way within your code. While Node.js functions are called marshall and unmarshall, in the Boto3 library for Python, similar ...