In Pydantic 2, you can use MyModel.model_validate(my_dict) to generate a model from a dictionary. According to the documentation –

this is very similar to the __init__ method of the model, except it takes a dict rather than keyword arguments.

If you're Pydantic 1, the method is parse_obj instead.

I haven't found a way to make the Globals class use the extended definition of the table

You can change the type of a field in a subclass if you declare the field again using the desired type.

Also, simply adding new fields to the subclass does not seem to work

It looks like you are setting instance attributes in the __init__() method, but fields are declared as class attributes.

This example shows a way to add a calculated field records_by_id to ExtendedTable and use ExtendedTable in ExtendedGlobals:

# File: extensions.py
import json
from typing import Any, Dict, List, Optional

from pydantic import Field, validator

from datamodel import Globals, Record, Table


class ExtendedTable(Table):
    # New fields are declared as class attributes not as instance attributes inside the __init__()
    # Calculated fields usually have a default value or default factory so that you don't have to provide a value
    # I prefer a default_factory for mutable values
    records_by_id: Dict[int, Record] = Field(default_factory=dict)

    # A validator can populate a calculated field
    # Use always=True to run the validator even if a value is not supplied and the default value is used
    @validator("records_by_id", always=True)
    def _calculate_records_by_id(
        cls, value: Dict[int, Record], values: Dict[str, Any]
    ) -> Dict[int, Record]:
        records: Optional[List[Record]] = values.get("records")
        if records is None:
            # The records field was not valid
            # Return value or raise a ValueError instead if you want
            return value
        return {record.id: record for record in records}


class ExtendedGlobals(Globals):
    # You can change the type of a field in a subclass if you declare the field again
    table: ExtendedTable


if __name__ == "__main__":
    records = """
    {
        "table": {
            "records": [{"id": 0, "name": "A"}, {"id": 1, "name": "B"}]
        }
    }
    """

    content = json.loads(records)
    extended_globals = ExtendedGlobals.parse_obj(content)
    print(repr(extended_globals))

Output:

ExtendedGlobals(table=ExtendedTable(records=[Record(id=0, name='A'), Record(id=1, name='B')], records_by_id={0: Record(id=0, name='A'), 1: Record(id=1, name='B')}))

This has been discussed some time ago and Samuel Colvin said he didn't want to pursue this as a feature for Pydantic.

If you are fine with code generation instead of actual runtime creation of models, you can use the datamodel-code-generator.

To be honest, I struggle to see the use case for generating complex models at runtime, seeing as their main purpose is validation, implying that you think about correct schema before running your program. But that is just my view.

For simple models I guess you can throw together your own logic for this fairly quickly.

If do you need something more sophisticated, the aforementioned library does offer some extensibility. You should be able to import and inherit from some of their classes like the JsonSchemaParser. Maybe that will get you somewhere.

Ultimately I think this becomes non-trivial very quickly, which is why Pydantic's maintainer didn't want to deal with it and why there is a whole separate project for this.