The write() method from SoftwareSerial (which is inherited from the
abstract Print class) does not support printing PROGMEM-based binary
buffers. The print() method does have some PROGMEM support, but it is
limited to printing NUL-terminated character strings, and is meant to be
used with the F() macro. You could coerce this method into printing
your data, but this would require you to add a null byte at the end of
each inner array, and the trick would break if you ever have to send a
null byte.
Also, the pgm_read_stuff() macros cannot read an array at a time. You
could do achieve this with memcpy_P(), but that would require
allocating an array in RAM. I think the simplest solution here is to get
the bytes from the flash one by one with pgm_read_byte():
const byte *message = messages_for_measurement[0];
size_t size = sizeof messages_for_measurement[0];
for (size_t i = 0; i < size; i++) {
swSerial.write(pgm_read_byte(&message[i]));
}
Note that the first line does not attempt to access the array data: it
only copies into message the address of the first byte: this is the
decay-to-pointer semantics.
I am afraid there is no good solution to this problem. One option I do
like is to use the __flash qualifier instead of PROGMEM:
const uint8_t ram_array[] = { 1, 2, 3, 4 };
__flash const uint8_t flash_array[] = { 5, 6, 7, 8 };
void function_reading_ram(const uint8_t *array)
{
uint8_t secondElement = array[1];
// ...
}
void function_reading_flash(__flash const uint8_t *array)
{
uint8_t secondElement = array[1];
// ...
}
int main(void)
{
function_reading_ram(ram_array); // OK
function_reading_flash(flash_array); // OK
function_reading_ram(flash_array); // Warning
function_reading_flash(ram_array); // Warning
}
Note that, with __flash, you don't need pgm_read_byte_near() or
anything similar. You just use the array like you would use any regular
array, and the compiler is smart enough to generate the code required to
access the Flash memory.
The warnings generated by gcc are:
warning: conversion from address space ‘__flash’ to address space ‘generic’ [-Waddr-space-convert]
function_reading_ram(flash_array);
^
warning: conversion from address space ‘generic’ to address space ‘__flash’ [-Waddr-space-convert]
function_reading_flash(ram_array);
^
Given how nice this solution is, you may wonder why I wrote “there is no
good solution”. Well, this comes with two caveats, one small and one
huge. The small caveat is that you need to explicitly pass the option
-Waddr-space-convert to the compiler if you want it to generate the warnings.
It is not implied even with
-Wall -Wextra. The huge caveat is that this only works in plain C.
If you try to use this trick in C++ you get:
error: ‘__flash’ does not name a type
This is an implementation of the “named address spaces” extension to the standard C. Since nothing like this is standardized in C++, the authors of gcc assumed it would not be useful to users of C++. :-( You can mix C and C++ sources in Arduino, but you can't call class methods from C.
I will expand on KIIV's comment here.
This is a known issue to the people who wrote the Arduino framework. Essentially, the pointers to PROGMEM and RAM are both data pointers (think of a pointer like an array index, but for the entire RAM/Flash itself rather than one chunk of it that you manage) of the same size and type. They cannot be told apart since pointer types are only determined by the size of the data, not where the data is.
So, they used a clever solution: Make the pointer into a different pointer that isn't compatible with the regular RAM pointer. This is done by casting to the __FlashStringHelper class (and then casting back when it must be read). The FSH class doesn't actually have any methods or do anything; it's simply used to define a custom pointer type that is different from the normal const *.
You can directly go read one of the better guides I've seen on it here (Note that it's for the ESP8266, but this part works the same), or I can try to explain it. I suggest reading it anyway, since it has a lot of other info that's useful but not relevant to this question. Do note that they have a few examples near the end which don't de-allocate memory when they are done using it, though, so copying them directly could cause a memory leak (see my string processing example to see the proper method).
First, you need to cast the array to the FSH class:
const byte MyProgmemArray[] PROGMEM = {4,4,4,4,4,4,4,4}; //Fixes issue #82 (add randomness); see https://xkcd.com/221/ for details :D
__FlashStringHelper * MyArray = (__FlashStringHelper *) MyProgmemArray;
I am unaware if there is any method (and I did look) to declare these in-place, i.e. to make the original array declaration into a __FlashStringHelper * directly. So you still need, as the programmer/user, to know which strings are in flash when you add them, and add the conversion line. This means that what you wanted (fully automatic RAM/Flash detection) isn't quite possible, but this is as close as I think it gets. A further warning is that the original array still exists, meaning that it could accidentally be used instead (will still cause the original mismatch issue) and the new definition takes up slightly more memory for the additional pointer (if the compiler does not optimize it out?).
Oddly, it works fine if your want to store a string:
__FlashStringHelper * MyString = PSTR("yaddayaddayadda...");
Strings can be inlined, but I think it's because of both the PSTR macro and because string definitions might be treated differently than an array of values (even though they are physically the same thing). Perhaps you could experiment with however PSTR works (it's actually a macro that does something, and isn't just a keyword) and see if you can implement this in your own code after all.
Now, you still need to get the data OUT of the array in that function you mentioned. To do that, we actually use two versions of that function. This is called overloading (and I think it's C++ only, but this is Arduino, which is by definition C++, so it's safe).
The RAM function you declare as normal:
void MyFunction(const uint8_t *MY_ARRAY){
byte index_two = MY_ARRAY[1];
...
}
The PROGMEM function you declare as follows (assuming that pgm_read_byte_near IS the right function. Also note that it gets more complicated if you need multi-byte values, like int or something...):
void MyFunction(__FlashStringHelper *MY_PROGMEM_ARRAY){
const byte *MY_ARRAY = (const byte *)MY_PROGMEM_ARRAY; //convert back to normal
byte index_two = pgm_read_byte_near(MY_ARRAY+1); //still have to read it as flash, of course
...
}
As a bonus bit of info that is unrelated to the question as asked, but that may interest you/be relevant later:
There are a few more notes if you use this for string data (or pretend it's a string for this part, which works if your data contains no zeros and ends in one). By the latter, I refer to the fact that strings in C (NOT the C++ std::string or Arduino String, though, so keep this in mind) are basically char arrays with a 0 tacked on the end. All string-processing functions run an infinite loop until they read a 0 at their current index, which allows them to work with strings of any length but still know when they are done. As a result, when reading it, you can treat your data like a string, with three caveats:
- It can never contain a 0, except for the very last item, which won't be read.
- When defining it, you'd need to still use {} instead of "" since most values don't have printable ASCII characters. (And even so, you'd have to write
" "for 32, for instance.) - You can't actually print the alleged string, since the data won't actually contain printable characters. You'll get any mix of numbers, letters, punctuation, and fun things like backspace and newline all jumbled together. You would need to read each character and convert it (via, say,
String num = <your value here>;oritoa()etc).
If you want to print it using an existing Arduino function, like Serial.print, and it is actually a string (and not data pretending to be a string), it will natively support __FlashStringHelper *, so you don't need the conversion step (this lets you do Serial.print(F("No static RAM used here!")); and similar).
There are two ways to read the data if it is for your own string-processing function, however. The first, the simple one, is not memory efficient and creates a full copy of the data or string in RAM while it's being used. As a result, it is not capable of processing strings that are too large for RAM and may have issues if it's called while a lot of RAM is already in use or it is called deep down a set of nested functions. It is very convenient, however:
//convert from __FlashStringHelper * as above, but let's assume it's a char* here
int size = strlen_P(MY_ARRAY);
if (size!=0){
char * data = new char[size];
if (data!=NULL){
strcpy_P(data,MY_ARRAY);
//do stuff with the string
delete data;
}
}
The second method is more annoying to actually do, and requires more control over how the data is processed, but you can also read out the data byte-by-byte as shown in my first decoding example above using pgm_read_byte_near. This lets you work with data that is too large to fit in RAM and keeps a constant stack size too. The downside is that it requires that you are actually able to stream the data byte-by-byte in all places that it's then used (or that you re-read it when it needs to be accessed out-of-order).