Use DataFrame.groupby with DataFrame.apply and DataFrame.to_dict all columns with no Col1 filtered by Index.difference, create DataFrame by DataFrame.reset_index and last use DataFrame.to_dict for dictionary output or DataFrame.to_json for json output:

cols = df.columns.difference(['Col1'])
d = (df.groupby('Col1')[cols]
        .apply(lambda x: x.to_dict('r'))
        .reset_index(name='Other_details')
        .to_dict(orient='records'))

cols = df.columns.difference(['Col1'])
d = (df.groupby('Col1')[cols]
        .apply(lambda x: x.to_dict('r'))
        .reset_index(name='Other_details')
        .to_json(orient='records'))
Answer from jezrael on Stack Overflow
🌐
Pandas
pandas.pydata.org › docs › reference › api › pandas.DataFrame.to_json.html
pandas.DataFrame.to_json — pandas 3.0.2 documentation
The string ‘index’ as a column name with empty Index or if it is ‘index’ will raise a ValueError. ... Length of whitespace used to indent each record. ... Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-v...
Discussions

python - Using column values as key in pandas json - Stack Overflow
I have a pandas dataframe as below. I'm just wondering if there's any way to have my column values as my key to the json. df: |symbol | price| |:------|------| |a. |120| |b. |100| |c ... More on stackoverflow.com
🌐 stackoverflow.com
How to obtained the json from the dataframe with the column names for every row as a key using python - Stack Overflow
I want the json data from the dataframe where I need to have the column name as key for every row using python. Name Age Qualification A 35 MCA B 30 Phd C 28 Msc... More on stackoverflow.com
🌐 stackoverflow.com
How to convert the row-wise data of dataframe with its column name as key and row data as value in json using python - Stack Overflow
I am having the issue in converting the row-wise data of dataframe with the column name as key and row data as value. I want to pass this row-wise json to another API as an input. I am unable to ge... More on stackoverflow.com
🌐 stackoverflow.com
September 27, 2019
python - DataFrame to Json Using First Col as Key and Second as Value - Stack Overflow
1 Pandas transform json column into multiple columns · 0 How to load JSON to Dataframe with key value pair as two colums More on stackoverflow.com
🌐 stackoverflow.com
🌐
GeeksforGeeks
geeksforgeeks.org › how-to-convert-pandas-dataframe-into-json-in-python
How to convert pandas DataFrame into JSON in Python? - GeeksforGeeks
June 12, 2025 - {"Name":{"0":"John","1":"Anna","2":"Peter"},"Age":{"0":28,"1":24,"2":35},"City":{"0":"New York","1":"Paris","2":"Berlin"}} This example shows how easy it is to convert a DataFrame into JSON format using the to_json() method where both columns ...
🌐
Pandas
pandas.pydata.org › docs › dev › reference › api › pandas.DataFrame.to_json.html
pandas.DataFrame.to_json — pandas documentation
The string ‘index’ as a column name with empty Index or if it is ‘index’ will raise a ValueError. ... Length of whitespace used to indent each record. ... Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-v...
🌐
Stack Overflow
stackoverflow.com › questions › 72259400 › using-column-values-as-key-in-pandas-json
python - Using column values as key in pandas json - Stack Overflow
df.groupby('symbol').price.apply(list).to_json() [Out]: {"a":[120],"b":[100],"c":[200]} ... Sign up to request clarification or add additional context in comments. ... import pandas as pd d = {'symbol': ['a', 'b', 'c'], 'price': [120, 100, 200]} df = pd.DataFrame(data=d) print(df) print (df.set_index('symbol').rename(columns={'price':'json_data'}).to_json()) # EXPORT TO FILE df.set_index('symbol').rename(columns={'price':'json_data'}).to_json('price.json')
🌐
Stack Overflow
stackoverflow.com › questions › 58135246 › how-to-convert-the-row-wise-data-of-dataframe-with-its-column-name-as-key-and-ro
How to convert the row-wise data of dataframe with its column name as key and row data as value in json using python - Stack Overflow
September 27, 2019 - import requests import pandas as pd def call_api(json_data): r = requests.post(url, headers=headers, data=out) print(r.json()) df.apply(lambda x:call_api(x.to_json()),axis=1) Hope this helps! ... It will convert the complete dataframe into the json.
Find elsewhere
Top answer
1 of 2
2

Pandas is equipped for this out of the box.

pandas.DataFrame.to_json

here is the example dataframe:

import json
df = pd.DataFrame(
    [["a", "b"], ["c", "d"]],
    index=["row 1", "row 2"],
    columns=["col 1", "col 2"],
)

Here is the result using to_json():

result = df.to_json(orient="split")
parsed = json.loads(result)
json.dumps(parsed, indent=4)  
{
    "columns": [
        "col 1",
        "col 2"
    ],
    "index": [
        "row 1",
        "row 2"
    ],
    "data": [
        [
            "a",
            "b"
        ],
        [
            "c",
            "d"
        ]
    ]
}

here is the link: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_json.html

2 of 2
1

As per the function provided here @Parsa T. You can just change the column names and use the function to get the required result.

def set_for_keys(my_dict, key_arr, val):
    """
    Set value at the path in my_dict defined by the string (or serializable object) array key_arr
    """
    current = my_dict
    for i in range(len(key_arr)):
        key = key_arr[i]
        if key not in current:
            current[key] = val if i==len(key_arr)-1 else {}
        else:
            if type(current[key]) is not dict:
                print("Given dictionary is not compatible with key structure requested")
                raise ValueError("Dictionary key already occupied")

        current = current[key]

    return my_dict

def to_formatted_json(df, sep="."):
    result = []
    for _, row in df.iterrows():
        parsed_row = {}
        for idx, val in row.iteritems():
            keys = idx.split(sep)
            parsed_row = set_for_keys(parsed_row, keys, val)

        result.append(parsed_row)
    return result


df.columns = ['ID', 'PERSONAL.NAME', 'PERSONAL.LAST', 'GEO.ADDRESS', 'GEO.COUNTY']
#Where df was parsed from json-dict using json_normalize
print(to_formatted_json(df, sep="."))

OUTPUT:

[{'ID': '0',
  'PERSONAL': {'NAME': 'jimmy', 'LAST': 'neutron'},
  'GEO': {'ADDRESS': '101 ocean avenue', 'COUNTY': 'yellow card park'}},
 {'ID': '1',
  'PERSONAL': {'NAME': 'james', 'LAST': 'baxter'},
  'GEO': {'ADDRESS': '202 bubble gum county', 'COUNTY': 'candy kingdom'}},
 {'ID': '2',
  'PERSONAL': {'NAME': 'joben', 'LAST': 'segel'},
  'GEO': {'ADDRESS': '303 china town', 'COUNTY': 'universal studio'}}]
🌐
datagy
datagy.io › home › pandas tutorials › pandas reading & writing data › convert a pandas dataframe to json
Convert a Pandas DataFrame to JSON • datagy
December 15, 2022 - By passing 'columns' into the Pandas .to_json() method’s orient argument, you return a JSON string that formats the data in the format of a dictionary that contains the columns as keys and dictionaries of the index to record mappings.
🌐
Saturn Cloud
saturncloud.io › blog › converting-pandas-dataframe-to-json-object-column-a-comprehensive-guide
Converting Pandas DataFrame to JSON Object Column: A Guide | Saturn Cloud Blog
January 4, 2024 - Finally, we can add the JSON object as a new column in the DataFrame. We use the apply() function to apply a function across the DataFrame’s rows. df['JSON_Object'] = df.apply(lambda row: json.dumps(row.to_dict()), axis=1) print(df) And that’s it! You have successfully converted a Pandas DataFrame to a JSON object column. ... Name Age Occupation JSON_Object 0 John 28 Engineer {"Name": "John", "Age": 28, "Occupation": "Engineer"} 1 Anna 24 Doctor {"Name": "Anna", "Age": 24, "Occupation": "Doctor"} 2 Peter 22 Student {"Name": "Peter", "Age": 22, "Occupation": "Student"}
🌐
Statology
statology.org › home › how to convert a pandas dataframe to json
How to Convert a Pandas DataFrame to JSON
July 30, 2020 - df.to_json(orient='columns') { "points": { "0": 25, "1": 12, "2": 15, "3": 19 }, "assists": { "0": 5, "1": 7, "2": 7, "3": 12 } } df.to_json(orient='values') [ [ 25, 5 ], [ 12, 7 ], [ 15, 7 ], [ 19, 12 ] ] df.to_json(orient='table') { "schema": { "fields": [ { "name": "index", "type": "integer" }, { "name": "points", "type": "integer" }, { "name": "assists", "type": "integer" } ], "primaryKey": [ "index" ], "pandas_version": "0.20.0" }, "data": [ { "index": 0, "points": 25, "assists": 5 }, { "index": 1, "points": 12, "assists": 7 }, { "index": 2, "points": 15, "assists": 7 }, { "index": 3, "points": 19, "assists": 12 } ] }
🌐
Data to Fish
datatofish.com › export-pandas-dataframe-json
How to Export a pandas DataFrame as a JSON File
After you run this, you should find a fish.json file on your desktop. Open it with a text editor and you should see the following content: ... { "columns":["fish","count"], "index":[0,1,2], "data":[["salmon",100],["pufferfish",10],["shark",1]] } ... { "0":{"fish":"salmon","count":100}, "1":{"fish":"pufferfish","count":10},"2":{"fish":"shark","count":1} } ... { "schema":{ "fields":[ {"name":"index","type":"integer"}, {"name":"fish","type":"string"}, {"name":"count","type":"integer"} ], "primaryKey":["index"],"pandas_version":"1.4.0"}, "data":[ {"index":0,"fish":"salmon","count":100}, {"index":1,"fish":"pufferfish","count":10}, {"index":2,"fish":"shark","count":1} ] }
Top answer
1 of 2
12

Use json.loads or ast.literal_eval for convert strings to list of dicts:

import ast, json

df = pd.DataFrame(rows) 
df['Sales_Plan_Details'] = df['Sales_Plan_Details'].apply(json.loads)
#alternative solution
#df['Sales_Plan_Details'] = df['Sales_Plan_Details'].apply(ast.literal_eval)

j = df.to_json(orient='records')
print (j)
[{"Sales_Plan_Details":[{"Month":"2019-1","Quantity":10,"Product_Gid":3}],
  "customer_name":"ABI2","employee_name":"ASU2","location_name":"Cherai2"},
{"Sales_Plan_Details":[{"Month":"2019-1","Quantity":10,"Product_Gid":3}],
 "customer_name":"ABI","employee_name":"ASU","location_name":"Cherai"}]

Setup:

rows= [{
                    "customer_name": "ABI2",
                    "location_name": "Cherai2",
                    "employee_name": "ASU2",
                    "Sales_Plan_Details": "[{\"Month\": \"2019-1\", \"Quantity\": 10, \"Product_Gid\": 3}]"

    },
{
                "customer_name": "ABI",
                "location_name": "Cherai",
                "employee_name": "ASU",
                "Sales_Plan_Details": "[{\"Month\": \"2019-1\", \"Quantity\": 10, \"Product_Gid\": 3}]"

}]
2 of 2
1

You can use list comprehensions to map the Sales_Plan_Details values.

You can use json.loads() to deserialize the list value from the string.

import json

dataframe_json = [
    {
                    "customer_name": "ABI2",
                    "location_name": "Cherai2",
                    "employee_name": "ASU2",
                    "Sales_Plan_Details": "[{\"Month\": \"2019-1\", \"Quantity\": 10, \"Product_Gid\": 3}]"

    },
    {
                    "customer_name": "ABI",
                    "location_name": "Cherai",
                    "employee_name": "ASU",
                    "Sales_Plan_Details": "[{\"Month\": \"2019-1\", \"Quantity\": 10, \"Product_Gid\": 3}]"

    }]

# get the "Sales_Plan_Details" key value's from the list
sales_plan_details_nested_list = [sales_plan_details_dict for sales_plan_details_dict in json.loads(item("Sales_Plan_Details")) for item in dataframe_json]

# flatten the list
sales_plan_details_list = [item for sublist in sales_plan_details_nested_list for item in sublist]

# pretty print the list now
print(json.dumps(sales_plan_details_list, indent=True))
🌐
Reddit
reddit.com › r/learnpython › using values of dataframe column as json keys
r/learnpython on Reddit: Using values of dataframe column as JSON Keys
January 18, 2024 - I have a pandas dataframe with 6 columns that I am trying to convert to JSON in a desired format. What I've been having trouble with is using the actual values of a column in my dataframe as keys in my JSON.
🌐
pandas
pandas.pydata.org › pandas-docs › dev › reference › api › pandas.DataFrame.to_json.html
pandas.DataFrame.to_json — pandas 3.0.0rc1+117.gff0cd9a3a7 documentation
The string ‘index’ as a column name with empty Index or if it is ‘index’ will raise a ValueError. ... Length of whitespace used to indent each record. ... Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-v...
🌐
Stack Overflow
stackoverflow.com › questions › 73807703 › how-to-place-json-key-values-into-columns-using-pandas-dataframe
python - How to place JSON key values into columns using Pandas Dataframe - Stack Overflow
you can store the key/value pairs as json strings in a column. you then can use the key/value pair later by access the contents of the column and loading the data into a dictionary · ListenSoftware Louise Ai Agent – ListenSoftware Louise ...
🌐
Vultr Docs
docs.vultr.com › python › third-party › pandas › DataFrame › to_json
Python Pandas DataFrame to_json() - Convert to JSON | Vultr Docs
December 31, 2024 - import pandas as pd # Sample DataFrame ... · In this example, the DataFrame df is converted to a JSON string. By default, the to_json() method uses the 'columns' orientation, where each column becomes a key, and their values are listed ...
Top answer
1 of 3
1

JSON files are treated as dicts in python, the JSON file you specified has duplicate keys and could only be parsed as a string (and not using the python json library). The following code:

import json
from io import StringIO

df = pd.DataFrame(np.arange(1,10).reshape((3,3)), columns=['col1','col2','col3'])
io = StringIO()
df.to_json(io, orient='columns')
parsed = json.loads(io.getvalue())
with open("pretty.json", '+w') as of:
    json.dump(parsed, of, indent=4)

will produce the following JSON:

{
    "col1": {
        "0": 1,
        "1": 4,
        "2": 7
    },
    "col2": {
        "0": 2,
        "1": 5,
        "2": 8
    },
    "col3": {
        "0": 3,
        "1": 6,
        "2": 9
    }
}

which you could later load to python. alternatively, this script will produce exatcly the string you want:

with open("exact.json", "w+") as of:
    of.write('[\n\t{\n' + '\t},\n\t{\n'.join(["".join(["\t\t\"%s\": %s,\n"%(c, df[c][i]) for i in df.index]) for c in df.columns])+'\t}\n]')

and the output would be:

[
    {
        "col1": 1,
        "col1": 4,
        "col1": 7,
    },
    {
        "col2": 2,
        "col2": 5,
        "col2": 8,
    },
    {
        "col3": 3,
        "col3": 6,
        "col3": 9,
    }
]

edit: fixed brackets

2 of 3
1

You need to do

df.to_json('file.json', orient='records')

Note that this will give you a array of objects:

[
        {
            "col1": 1,
            "col1": 4,
            "col1": 7
        },
        {
            "col2": 2,
            "col2": 5,
            "col2": 8
        },
        {
            "col3": 3,
            "col3": 6,
            "col3": 9
        }
]

You can also do

df.to_json('file.json', orient='records', lines=True)

if you want output like:

{"col1":1,"col1":4,"col1":7},
{"col2":2,"col2":5,"col2":8},
{"col3":3,"col3":6,"col3":9}

To prettify output:

pip install jq
cat file.json | jq '.' > new_file.json