It isn't possible to dynamically define a struct that is identical to a compile-time struct.
It is possible, but difficult, to create dynamic structures that can contain the information equivalent to a struct. The access to the data is less convenient than what is available at compile-time.
All else apart, you cannot access a member somestruct.not_seen_at_compile_time using the dot . or arrow -> notation if it was not defined at compile-time.
With network communications, there are other issues to address - notably 'endianness'. That is, the data on the wire will probably include multi-byte (2, 4, 8) integers, and either the MSB or the LSB will be sent first, but if one machine is little-endian (IA-32, IA-64, x86/64) and the other is big-endian (SPARC, PPC, almost anything not from Intel), then the data will need to be transformed. Floating-point formats can also be problematic. There are numerous standards dedicated to defining how data will be sent across the network - it is not trivial. Some are specific: IP, TCP, UDP; others are general, such as ASN.1.
However, the 'cannot do dynamic data structures' part limits things - you have to agree beforehand on what the data structures are, and how they will be interpreted.
How do you do that?
gerty3000 asks:
It is possible, but difficult, to create dynamic structures that can contain the information equivalent to a struct. — How do you do that? I would like to pass dynamically-defined structs off to other C code (assume same compiler and other settings) without having to duplicate the struct memory layout routines from the compiler. I won't be accessing fields of these structs inside my process much (just initializing them once), so convenient syntax is not a concern.
You can't do it without duplicating the memory layout in some shape or form. It might not have to be exactly the same, but it is likely best if it is. Here's some sample code that shows roughly how it might be done.
dynstruct.c
This contains the basic structure manipulation material — structures to describe structures and (simple) members. Handling full arrays (as opposed to strings) would require more work, and there's a good deal of make-work replication to be managed for other types.
It also contains a main() program that tests the code. It makes a call to other_function(), which demonstrates that the structure I've defined in the data structures does match the structure exactly. The data does assume a 64-bit machine where double must be aligned on an 8-byte boundary (so there's a 4-byte hole in the structure); you will have to tweak the data for a machine where double can be on a 4-byte boundary.
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
/* This is the type that will be simulated dynamically */
/*
struct simulated
{
int number;
double value;
char string[32];
};
*/
/* SOF structure.h */
typedef enum Type { INT, DOUBLE, STRING } Type;
typedef struct Descriptor
{
size_t offset;
Type type;
size_t type_size;
size_t array_dim;
char name[32];
} Descriptor;
typedef struct Structure
{
size_t size;
char name[32];
Descriptor *details;
} Structure;
extern void *allocate_structure(const Structure *structure);
extern void deallocate_structure(void *structure);
extern void *pointer_to_element(void *p, const Descriptor *d);
extern int get_int_element(void *p, const Descriptor *d);
extern void set_int_element(void *p, const Descriptor *d, int newval);
extern double get_double_element(void *p, const Descriptor *d);
extern void set_double_element(void *p, const Descriptor *d, double newval);
extern char *get_string_element(void *p, const Descriptor *d);
extern void set_string_element(void *p, const Descriptor *d, char *newval);
/* EOF structure.h */
static Descriptor details[] =
{
{ 0, INT, sizeof(int), 1, "number" },
{ 8, DOUBLE, sizeof(double), 1, "value" },
{ 16, STRING, sizeof(char), 32, "string" },
};
static Structure simulated = { 48, "simulated", details };
void *allocate_structure(const Structure *structure)
{
void *p = calloc(1, structure->size);
return p;
}
void deallocate_structure(void *structure)
{
free(structure);
}
void *pointer_to_element(void *p, const Descriptor *d)
{
void *data = (char *)p + d->offset;
return data;
}
int get_int_element(void *p, const Descriptor *d)
{
assert(d->type == INT);
int *v = pointer_to_element(p, d);
return *v;
}
void set_int_element(void *p, const Descriptor *d, int newval)
{
assert(d->type == INT);
int *v = pointer_to_element(p, d);
*v = newval;
}
double get_double_element(void *p, const Descriptor *d)
{
assert(d->type == DOUBLE);
double *v = pointer_to_element(p, d);
return *v;
}
void set_double_element(void *p, const Descriptor *d, double newval)
{
assert(d->type == DOUBLE);
double *v = pointer_to_element(p, d);
*v = newval;
}
char *get_string_element(void *p, const Descriptor *d)
{
assert(d->type == STRING);
char *v = pointer_to_element(p, d);
return v;
}
void set_string_element(void *p, const Descriptor *d, char *newval)
{
assert(d->type == STRING);
assert(d->array_dim > 1);
size_t len = strlen(newval);
if (len > d->array_dim)
len = d->array_dim - 1;
char *v = pointer_to_element(p, d);
memmove(v, newval, len);
v[len] = '\0';
}
extern void other_function(void *p);
int main(void)
{
void *sp = allocate_structure(&simulated);
if (sp != 0)
{
set_int_element(sp, &simulated.details[0], 37);
set_double_element(sp, &simulated.details[1], 3.14159);
set_string_element(sp, &simulated.details[2], "Absolute nonsense");
printf("Main (before):\n");
printf("Integer: %d\n", get_int_element(sp, &simulated.details[0]));
printf("Double: %f\n", get_double_element(sp, &simulated.details[1]));
printf("String: %s\n", get_string_element(sp, &simulated.details[2]));
other_function(sp);
printf("Main (after):\n");
printf("Integer: %d\n", get_int_element(sp, &simulated.details[0]));
printf("Double: %f\n", get_double_element(sp, &simulated.details[1]));
printf("String: %s\n", get_string_element(sp, &simulated.details[2]));
deallocate_structure(sp);
}
return 0;
}
other.c
This code knows nothing about the structure description material in dynstruct.c; it knows about the struct simulated that the simulation code simulates. It prints the data it is passed and modifies it.
#include <stdio.h>
#include <string.h>
extern void other_function(void *p);
struct simulated
{
int number;
double value;
char string[32];
};
void other_function(void *p)
{
struct simulated *s = (struct simulated *)p;
printf("Other function:\n");
printf("Integer: %d\n", s->number);
printf("Double: %f\n", s->value);
printf("String: %s\n", s->string);
s->number *= 2;
s->value /= 2;
strcpy(s->string, "Codswallop");
}
Sample output
Main (before):
Integer: 37
Double: 3.141590
String: Absolute nonsense
Other function:
Integer: 37
Double: 3.141590
String: Absolute nonsense
Main (after):
Integer: 74
Double: 1.570795
String: Codswallop
Clearly, this code is not production-ready. It is a sufficient demonstration of what can be done. One issue you'd have to deal with is initializing the Structure and Descriptor data correctly. You can't put too many assertions into that sort of code. For example, I should really have assert(d->size == sizeof(double); in get_double_element(). It would also be sensible to include assert(d->offset % sizeof(double) == 0); to ensure that the double element is properly aligned. Or you might have a validate_structure(const Structure *sp); function that did all these validation checks. You'd need a function void dump_structure(FILE *fp, const char *tag, const Structure *sp); to dump the defined structure to the given file preceded by the tag, to assist in debugging. Etc.
This code is pure C; it is not compilable by a C++ compiler as C++. There aren't enough casts to satisfy a C++ compiler.
Answer from Jonathan Leffler on Stack OverflowIt isn't possible to dynamically define a struct that is identical to a compile-time struct.
It is possible, but difficult, to create dynamic structures that can contain the information equivalent to a struct. The access to the data is less convenient than what is available at compile-time.
All else apart, you cannot access a member somestruct.not_seen_at_compile_time using the dot . or arrow -> notation if it was not defined at compile-time.
With network communications, there are other issues to address - notably 'endianness'. That is, the data on the wire will probably include multi-byte (2, 4, 8) integers, and either the MSB or the LSB will be sent first, but if one machine is little-endian (IA-32, IA-64, x86/64) and the other is big-endian (SPARC, PPC, almost anything not from Intel), then the data will need to be transformed. Floating-point formats can also be problematic. There are numerous standards dedicated to defining how data will be sent across the network - it is not trivial. Some are specific: IP, TCP, UDP; others are general, such as ASN.1.
However, the 'cannot do dynamic data structures' part limits things - you have to agree beforehand on what the data structures are, and how they will be interpreted.
How do you do that?
gerty3000 asks:
It is possible, but difficult, to create dynamic structures that can contain the information equivalent to a struct. — How do you do that? I would like to pass dynamically-defined structs off to other C code (assume same compiler and other settings) without having to duplicate the struct memory layout routines from the compiler. I won't be accessing fields of these structs inside my process much (just initializing them once), so convenient syntax is not a concern.
You can't do it without duplicating the memory layout in some shape or form. It might not have to be exactly the same, but it is likely best if it is. Here's some sample code that shows roughly how it might be done.
dynstruct.c
This contains the basic structure manipulation material — structures to describe structures and (simple) members. Handling full arrays (as opposed to strings) would require more work, and there's a good deal of make-work replication to be managed for other types.
It also contains a main() program that tests the code. It makes a call to other_function(), which demonstrates that the structure I've defined in the data structures does match the structure exactly. The data does assume a 64-bit machine where double must be aligned on an 8-byte boundary (so there's a 4-byte hole in the structure); you will have to tweak the data for a machine where double can be on a 4-byte boundary.
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
/* This is the type that will be simulated dynamically */
/*
struct simulated
{
int number;
double value;
char string[32];
};
*/
/* SOF structure.h */
typedef enum Type { INT, DOUBLE, STRING } Type;
typedef struct Descriptor
{
size_t offset;
Type type;
size_t type_size;
size_t array_dim;
char name[32];
} Descriptor;
typedef struct Structure
{
size_t size;
char name[32];
Descriptor *details;
} Structure;
extern void *allocate_structure(const Structure *structure);
extern void deallocate_structure(void *structure);
extern void *pointer_to_element(void *p, const Descriptor *d);
extern int get_int_element(void *p, const Descriptor *d);
extern void set_int_element(void *p, const Descriptor *d, int newval);
extern double get_double_element(void *p, const Descriptor *d);
extern void set_double_element(void *p, const Descriptor *d, double newval);
extern char *get_string_element(void *p, const Descriptor *d);
extern void set_string_element(void *p, const Descriptor *d, char *newval);
/* EOF structure.h */
static Descriptor details[] =
{
{ 0, INT, sizeof(int), 1, "number" },
{ 8, DOUBLE, sizeof(double), 1, "value" },
{ 16, STRING, sizeof(char), 32, "string" },
};
static Structure simulated = { 48, "simulated", details };
void *allocate_structure(const Structure *structure)
{
void *p = calloc(1, structure->size);
return p;
}
void deallocate_structure(void *structure)
{
free(structure);
}
void *pointer_to_element(void *p, const Descriptor *d)
{
void *data = (char *)p + d->offset;
return data;
}
int get_int_element(void *p, const Descriptor *d)
{
assert(d->type == INT);
int *v = pointer_to_element(p, d);
return *v;
}
void set_int_element(void *p, const Descriptor *d, int newval)
{
assert(d->type == INT);
int *v = pointer_to_element(p, d);
*v = newval;
}
double get_double_element(void *p, const Descriptor *d)
{
assert(d->type == DOUBLE);
double *v = pointer_to_element(p, d);
return *v;
}
void set_double_element(void *p, const Descriptor *d, double newval)
{
assert(d->type == DOUBLE);
double *v = pointer_to_element(p, d);
*v = newval;
}
char *get_string_element(void *p, const Descriptor *d)
{
assert(d->type == STRING);
char *v = pointer_to_element(p, d);
return v;
}
void set_string_element(void *p, const Descriptor *d, char *newval)
{
assert(d->type == STRING);
assert(d->array_dim > 1);
size_t len = strlen(newval);
if (len > d->array_dim)
len = d->array_dim - 1;
char *v = pointer_to_element(p, d);
memmove(v, newval, len);
v[len] = '\0';
}
extern void other_function(void *p);
int main(void)
{
void *sp = allocate_structure(&simulated);
if (sp != 0)
{
set_int_element(sp, &simulated.details[0], 37);
set_double_element(sp, &simulated.details[1], 3.14159);
set_string_element(sp, &simulated.details[2], "Absolute nonsense");
printf("Main (before):\n");
printf("Integer: %d\n", get_int_element(sp, &simulated.details[0]));
printf("Double: %f\n", get_double_element(sp, &simulated.details[1]));
printf("String: %s\n", get_string_element(sp, &simulated.details[2]));
other_function(sp);
printf("Main (after):\n");
printf("Integer: %d\n", get_int_element(sp, &simulated.details[0]));
printf("Double: %f\n", get_double_element(sp, &simulated.details[1]));
printf("String: %s\n", get_string_element(sp, &simulated.details[2]));
deallocate_structure(sp);
}
return 0;
}
other.c
This code knows nothing about the structure description material in dynstruct.c; it knows about the struct simulated that the simulation code simulates. It prints the data it is passed and modifies it.
#include <stdio.h>
#include <string.h>
extern void other_function(void *p);
struct simulated
{
int number;
double value;
char string[32];
};
void other_function(void *p)
{
struct simulated *s = (struct simulated *)p;
printf("Other function:\n");
printf("Integer: %d\n", s->number);
printf("Double: %f\n", s->value);
printf("String: %s\n", s->string);
s->number *= 2;
s->value /= 2;
strcpy(s->string, "Codswallop");
}
Sample output
Main (before):
Integer: 37
Double: 3.141590
String: Absolute nonsense
Other function:
Integer: 37
Double: 3.141590
String: Absolute nonsense
Main (after):
Integer: 74
Double: 1.570795
String: Codswallop
Clearly, this code is not production-ready. It is a sufficient demonstration of what can be done. One issue you'd have to deal with is initializing the Structure and Descriptor data correctly. You can't put too many assertions into that sort of code. For example, I should really have assert(d->size == sizeof(double); in get_double_element(). It would also be sensible to include assert(d->offset % sizeof(double) == 0); to ensure that the double element is properly aligned. Or you might have a validate_structure(const Structure *sp); function that did all these validation checks. You'd need a function void dump_structure(FILE *fp, const char *tag, const Structure *sp); to dump the defined structure to the given file preceded by the tag, to assist in debugging. Etc.
This code is pure C; it is not compilable by a C++ compiler as C++. There aren't enough casts to satisfy a C++ compiler.
No, it isn't in C all data types must be known at compile time. That's what makes it "real fast".
Best practice on allocating memory for a struct that has dynamic array members
c - How can I create a dynamically sized array of structs? - Stack Overflow
How to create a struct memory layout dynamically?
c++ struct, how to create struct objects dynamically, - Stack Overflow
Videos
Hi all! I am very new to C but am familiar with Python & Go quite a bit. Memory allocation is a new concept to me, but I want to start with best practices.
Lets say I want to create a struct that has some members, where one of them happens to be a pointer which I will size dynamically as an array.
Here is what I am doing in my init_my_struct(int n) function. I want to understand if I am doing something that is bad practice here
#include <stdlib.h>
#include <stdio.h>
struct my_struct {
int n;
int *my_arr; // this will be dynamically grown based on user arg
};
struct my_struct * init_my_struct(int n)
{
// allocate memory for one struct
struct my_struct *ms = malloc(sizeof(struct my_struct));
// use -> syntax since ms is a pointer
ms->n = n;
ms->my_arr = malloc(sizeof(int)*n);
return ms;
}
int main(int argc, char *argv[])
{
if (argc < 2) return 1;
int n = atoi(argv[1]);
// initialize the struct members
struct my_struct *ms = init_my_struct(n);
// check out contents of my struct
printf("my struct's n: %d\n", ms->n);
printf("my struct's my_arr: \n");
for (int i = 0; i < n; i++) {
printf("%d\n", ms->my_arr[i]);
}
free(ms); // clean up the memory
return 0;
}Any pointers or tips would be great! Thank you!
You've tagged this as C++ as well as C.
If you're using C++ things are a lot easier. The standard template library has a template called vector which allows you to dynamically build up a list of objects.
#include <stdio.h>
#include <vector>
typedef std::vector<char*> words;
int main(int argc, char** argv) {
words myWords;
myWords.push_back("Hello");
myWords.push_back("World");
words::iterator iter;
for (iter = myWords.begin(); iter != myWords.end(); ++iter) {
printf("%s ", *iter);
}
return 0;
}
If you're using C things are a lot harder, yes malloc, realloc and free are the tools to help you. You might want to consider using a linked list data structure instead. These are generally easier to grow but don't facilitate random access as easily.
#include <stdio.h>
#include <stdlib.h>
typedef struct s_words {
char* str;
struct s_words* next;
} words;
words* create_words(char* word) {
words* newWords = malloc(sizeof(words));
if (NULL != newWords){
newWords->str = word;
newWords->next = NULL;
}
return newWords;
}
void delete_words(words* oldWords) {
if (NULL != oldWords->next) {
delete_words(oldWords->next);
}
free(oldWords);
}
words* add_word(words* wordList, char* word) {
words* newWords = create_words(word);
if (NULL != newWords) {
newWords->next = wordList;
}
return newWords;
}
int main(int argc, char** argv) {
words* myWords = create_words("Hello");
myWords = add_word(myWords, "World");
words* iter;
for (iter = myWords; NULL != iter; iter = iter->next) {
printf("%s ", iter->str);
}
delete_words(myWords);
return 0;
}
Yikes, sorry for the worlds longest answer. So WRT to the "don't want to use a linked list comment":
#include <stdio.h>
#include <stdlib.h>
typedef struct {
char** words;
size_t nWords;
size_t size;
size_t block_size;
} word_list;
word_list* create_word_list(size_t block_size) {
word_list* pWordList = malloc(sizeof(word_list));
if (NULL != pWordList) {
pWordList->nWords = 0;
pWordList->size = block_size;
pWordList->block_size = block_size;
pWordList->words = malloc(sizeof(char*)*block_size);
if (NULL == pWordList->words) {
free(pWordList);
return NULL;
}
}
return pWordList;
}
void delete_word_list(word_list* pWordList) {
free(pWordList->words);
free(pWordList);
}
int add_word_to_word_list(word_list* pWordList, char* word) {
size_t nWords = pWordList->nWords;
if (nWords >= pWordList->size) {
size_t newSize = pWordList->size + pWordList->block_size;
void* newWords = realloc(pWordList->words, sizeof(char*)*newSize);
if (NULL == newWords) {
return 0;
} else {
pWordList->size = newSize;
pWordList->words = (char**)newWords;
}
}
pWordList->words[nWords] = word;
++pWordList->nWords;
return 1;
}
char** word_list_start(word_list* pWordList) {
return pWordList->words;
}
char** word_list_end(word_list* pWordList) {
return &pWordList->words[pWordList->nWords];
}
int main(int argc, char** argv) {
word_list* myWords = create_word_list(2);
add_word_to_word_list(myWords, "Hello");
add_word_to_word_list(myWords, "World");
add_word_to_word_list(myWords, "Goodbye");
char** iter;
for (iter = word_list_start(myWords); iter != word_list_end(myWords); ++iter) {
printf("%s ", *iter);
}
delete_word_list(myWords);
return 0;
}
If you want to dynamically allocate arrays, you can use malloc from stdlib.h.
If you want to allocate an array of 100 elements using your words struct, try the following:
words* array = (words*)malloc(sizeof(words) * 100);
The size of the memory that you want to allocate is passed into malloc and then it will return a pointer of type void (void*). In most cases you'll probably want to cast it to the pointer type you desire, which in this case is words*.
The sizeof keyword is used here to find out the size of the words struct, then that size is multiplied by the number of elements you want to allocate.
Once you are done, be sure to use free() to free up the heap memory you used in order to prevent memory leaks:
free(array);
If you want to change the size of the allocated array, you can try to use realloc as others have mentioned, but keep in mind that if you do many reallocs you may end up fragmenting the memory. If you want to dynamically resize the array in order to keep a low memory footprint for your program, it may be better to not do too many reallocs.
I am looking to create general tool that can load a database table and then save the table to binary file that can be read by c++. The c++ reader loads into a vector of structs. Each row in the vector represents row in the table.
So for example table could be int field double field
Then I want to in program dynamically some how create a struct with these fields and types and have vector of these and then save to binary file.
It has to be dynamic as tool takes on command line the table name and the fields.
Note the writer and reader will all be on the same os and same version of gcc.
Seems I would need to have some library that would tell me the memory layout of a struct in gcc. Any tools for this ?
First off, you should never try to generate variable names. it's just not the way to do it.
You should use a container (like in your example a std::vector) which is exactly what you want : something to put several other things of the same type.
In your example :
struct MyStruct { // Don't name your struct in all caps.
// it's generally reserved for MACROS. Use CamelCase
// blablabla
}
Do not use pointers unless you need to, usually it is much easier and much more efficient to use values. If you need to pass one of the elements of your vector to a function, so this function can modify it, you can pass an iterator to that element.
vector<MyStruct> mystructs;
for(int i = 0; i < 100; i++){
mystructs.emplace_back(MyStruct{});
}
Also if your struct has fields that need to be initialized, you might want to use an initializer list or define a constructor.
If you need to use dynamic allocation you can do something like this :
vector<MyStruct*> mystructs;
for(int i = 0; i < 100; i++){
mystructs.emplace_back(new MyStruct{});
}
Now the vector contains the pointers, not the objects. Each object is allocated individually in his own memory emplacement, therefore there is no garantee that they are next to each other. This matters tremendously for performance : Pointers give you indirection. Indirection leads to pain. Pain leads to anger ... blablabla => Welcome to the dark side.
A struct is a datatype that you defined, when you do MYSTRUCT *mystruct you declare a pointer to a MYSTRUCT object and the pointer is called mystruct It is the same thing you do with int *pointer_to_int.
For you first version, you redeclare mystruct at each iteration while your second declares two independant variables. If you want to declare a set of MYSTRUCT* easily, use an array of MYSTRUCT*: MYSTRUCT* mystruct_pointers_array[100].
NOTE MYSTRUCT* pointers are different than MYSTRUCT variables, you need to dereference the pointer to access to MYSTRUCT elements. E.g. mystruct->x.