First, a note on terminology: You appear to be talking about byte slices not byte arrays. Arrays in Go are of fixed length, so calling append on them won't work. See here for more.

Now to your question...

I am trying to make a for loop that will convert any data type to a byte array...

This is not possible, except in the most superficial, meaningless sense. For example, if you have a variable of type net.Conn, converting it to bytes will give you a meaningless value, since it is only meaningful in conjunction with one specific, active network connection.

However, assuming you don't have variables that refer to ephemeral state like this, the simplest way to convert arbitrary variables into a byte slice is with something like gob encoding.

However, this also has limitations, in that when serializing structs, it can only access exported fields, or those exposed through the GobEncoder interface. This Q&A explains why this limitation exists.

Now, in some cases you can work around this limitation with reflection, for example, but doing so is very cumbersome, and is usually unwise, since unexported fields often reference ephemeral state (see the note above).

So in summary: It's impossible to meaningfully do this for all data types. It's difficult to do it for many data types, and it's trivial to do it for "normal" data types.

Answer from Jonathan Hall on Stack Overflow
Discussions

Struct to byte[] vice versa

https://golang.org/pkg/encoding/gob/ this package is probably what you want mate :)

More on reddit.com
🌐 r/golang
18
16
December 3, 2020
go - How to convert type to byte array golang - Stack Overflow
How to Convert Type of one kind to byte array Here is the working example package main import ( "bytes" "fmt" "reflect" ) type Signature [5]byte const (... More on stackoverflow.com
🌐 stackoverflow.com
March 15, 2022
go - Convert []string to []byte - Stack Overflow
I am looking to convert a string array to a byte array in GO so I can write it down to a disk. What is an optimal solution to encode and decode a string array ([]string) to a byte array ([]byte)? ... More on stackoverflow.com
🌐 stackoverflow.com
go - Convert arbitrary Golang interface to byte array - Stack Overflow
This could have unexpected results ... play.golang.org/p/EgjvrqOyxEi 2019-10-14T16:13:52.183Z+00:00 ... it's worth to mention in your answer what @PassKit said in his comment 2020-01-21T23:26:54.713Z+00:00 ... Find the answer to your question by asking. Ask question ... See similar questions with these tags. ... 3 Unmarshalling array of bytes to an interface ... More on stackoverflow.com
🌐 stackoverflow.com
Top answer
1 of 11
51

One possible solution is the "encoding/gob" standard package. The gob package creates an encoder/decoder that can encode any struct into an array of bytes and then decode that array back into a struct. There's a great post, here.

As others have pointed out, it's necessary to use a package like this because structs, by their nature, have unknown sizes and cannot be converted into arrays of bytes.

I've included some code and a play.

package main

import (
    "bytes"
    "encoding/gob"
    "fmt"
    "log"
)

type P struct {
    X, Y, Z int
    Name    string
}

type Q struct {
    X, Y *int32
    Name string
}

func main() {
    // Initialize the encoder and decoder.  Normally enc and dec would be
    // bound to network connections and the encoder and decoder would
    // run in different processes.
    var network bytes.Buffer        // Stand-in for a network connection
    enc := gob.NewEncoder(&network) // Will write to network.
    dec := gob.NewDecoder(&network) // Will read from network.
    // Encode (send) the value.
    err := enc.Encode(P{3, 4, 5, "Pythagoras"})
    if err != nil {
        log.Fatal("encode error:", err)
    }

    // HERE ARE YOUR BYTES!!!!
    fmt.Println(network.Bytes())

    // Decode (receive) the value.
    var q Q
    err = dec.Decode(&q)
    if err != nil {
        log.Fatal("decode error:", err)
    }
    fmt.Printf("%q: {%d,%d}\n", q.Name, *q.X, *q.Y)
}
2 of 11
42

I assume you want something like the way C handles this. There is no built in way to do that. You will have to define your own serialization and deserialization to and from bytes for your struct. The binary package will help you encode the fields in your struct to bytes that you can add to the byte array but you will be responsible for specifying the lengths and offsets in the byte array that will hold the fields from your struct.

Your other options are to use one of the encoding packages: http://golang.org/pkg/encoding/ such as gob or json.

EDIT:

Since you want this for making a hash as you say in your comment the easisest thing to do is use []byte(fmt.Sprintf("%v", struct)) like so: http://play.golang.org/p/yY8mSdZ_kf

🌐
GitHub
gist.github.com › ecoshub › 5be18dc63ac64f3792693bb94f00662f
golang integer to byte array and byte array to integer function · GitHub
I wrote this not too long ago. Also as far as I know none of the examples above are specifically little- or big endian. This depends on your CPU and could (in theory) result in a big-endian byte array.
🌐
YourBasic
yourbasic.org › golang › convert-string-to-byte-slice
Convert between byte array/slice and string · YourBasic Go
yourbasic.org/golang · Basics · Convert string to bytes · Convert bytes to string · Performance · When you convert between a string and a byte slice (array), you get a brand new slice that contains the same bytes as the string, and vice versa. The conversion doesn’t change the data; the only difference is that strings are immutable, while byte slices can be modified.
🌐
Delft Stack
delftstack.com › home › howto › go › golang string to byte array
How to Convert String to Byte Array in Golang | Delft Stack
February 12, 2024 - Use the byte() function in Golang to convert a String to a Byte array. A byte is an unsigned 8-bit integer.
🌐
Medium
medium.com › @maximgradan › serialize-a-struct-to-bytes-to-send-it-through-the-network-in-go-part-i-b861c15d2f06
Serialize a struct to bytes to send it through the network in Go — Part I | by Maxim Gradan | Medium
June 2, 2022 - The pointer is just a number, an address in the RAM, whereas an array is a value and will entirely occupy the struct it belongs to no matter the array size. Now let’s see how the unsafe cast actually looks: Here we just take an address of the struct and interpret it as a byte array pointer then turn it into a slice.
Find elsewhere
🌐
Bacancy Technology
bacancytechnology.com › qanda › golang › assign-string-to-bytes-array
Mastering Go: How to Assign String to Bytes Array
December 13, 2023 - In this example, a byte array byteArray of the same length as the string is created. The copy() function is used to copy the bytes from the string to the byte array. If you want to work with the string at a character level and convert each character ...
🌐
GitHub
gist.github.com › miguelmota › 5bfa2b6ab88f439fe0da0bfb1faca763
Golang interface to bytes using gob encoder · GitHub
This only works if the inputted interface was originally bytes. If this were to be a string, etc... you would receive something like this from a panic: I solve this by converting the interface{} to string and then to []byte :
Top answer
1 of 8
31

Lets ignore the fact that this is Go for a second. The first thing you need is a serialization format to marshal the []string into.

There are many option here. You could build your own or use a library. I am going to assume you don't want to build your own and jump to serialization formats go supports.

In all examples, data is the []string and fp is the file you are reading/writing to. Errors are being ignored, check the returns of functions to handle errors.

Gob

Gob is a go only binary format. It should be relatively space efficient as the number of strings increases.

enc := gob.NewEncoder(fp)
enc.Encode(data)

Reading is also simple

var data []string
dec := gob.NewDecoder(fp)
dec.Decode(&data)

Gob is simple and to the point. However, the format is only readable with other Go code.

Json

Next is json. Json is a format used just about everywhere. This format is just as easy to use.

enc := json.NewEncoder(fp)
enc.Encode(data)

And for reading:

var data []string
dec := json.NewDecoder(fp)
dec.Decode(&data)

XML

XML is another common format. However, it has pretty high overhead and not as easy to use. While you could just do the same you did for gob and json, proper xml requires a root tag. In this case, we are using the root tag "Strings" and each string is wrapped in an "S" tag.

type Strings struct {
    S []string
}

enc := xml.NewEncoder(fp)
enc.Encode(Strings{data})

var x Strings
dec := xml.NewDecoder(fp)
dec.Decode(&x)
data := x.S

CSV

CSV is different from the others. You have two options, use one record with n rows or n records with 1 row. The following example uses n records. It would be boring if I used one record. It would look too much like the others. CSV can ONLY hold strings.

enc := csv.NewWriter(fp)
for _, v := range data {
    enc.Write([]string{v})
}
enc.Flush()

To read:

var err error
var data string
dec := csv.NewReader(fp)
for err == nil {        // reading ends when an error is reached (perhaps io.EOF)
    var s []string

    s, err = dec.Read()
    if len(s) > 0 {
        data = append(data, s[0])
    }
}

Which format you use is a matter of preference. There are many other possible encodings that I have not mentioned. For example, there is an external library called bencode. I don't personally like bencode, but it works. It is the same encoding used by bittorrent metadata files.

If you want to make your own encoding, encoding/binary is a good place to start. That would allow you to make the most compact file possible, but I hardly thing it is worth the effort.

2 of 8
12

The gob package will do this for you http://godoc.org/encoding/gob

Example to play with http://play.golang.org/p/e0FEZm-qiS

same source code is below.

package main

import (
    "bytes"
    "encoding/gob"
    "fmt"
)

func main() {
    // store to byte array
    strs := []string{"foo", "bar"}
    buf := &bytes.Buffer{}
    gob.NewEncoder(buf).Encode(strs)
    bs := buf.Bytes()
    fmt.Printf("%q", bs)

    // Decode it back
    strs2 := []string{}
    gob.NewDecoder(buf).Decode(&strs2)
    fmt.Printf("%v", strs2)
}
🌐
Reddit
reddit.com › r/golang › idiomatic way to "cast" []float32 to []byte
r/golang on Reddit: Idiomatic way to "cast" []float32 to []byte
September 19, 2023 -

Hi, I have an bitpack algorithm which needs a []byte slice e.g bitPack([]byte in, []byte out), my data is a float32 slice which I want to pack

Currently I'm doing the following:

func ToByte(in []float32) []byte {  
    return unsafe.Slice((*byte)(unsafe.Pointer(&in[0])), len(in)*4)  
}  

is this safe regarding GC, etc..., go vet at least does not complain.

🌐
GitHub
gist.github.com › SteveBate › 042960baa7a4795c3565
Example of converting a struct to a byte slice, compressing with gzip, and saving to file followed by the reverse process back to a struct · GitHub
Example of converting a struct to a byte slice, compressing with gzip, and saving to file followed by the reverse process back to a struct - struct_to_bytes.go
🌐
TutorialsPoint
tutorialspoint.com › golang-program-to-convert-file-to-byte-array
Golang program to convert file to byte array
February 21, 2023 - In Go programming language we can use byte function and ioutil.ReadFile function to convert a file into byte array. The os package file type that offers ways to open, read from, write to, and manipulate files whereas array is a fixed-size group of
🌐
TutorialsPoint
tutorialspoint.com › article › golang-program-to-convert-string-value-to-byte-value
Golang Program to Convert String Value to Byte Value
October 7, 2023 - Finally we print the slice of byte values on the screen using the function fmt.Println () which formats using the default formats for its operands and writes to standard output · In the above two examples, we have successfully complied and executed the golang program code to convert the string values to byte values.
🌐
GitHub
gist.github.com › teocci › 726a6331cb46a530ce007519df8459f0
Golang : Convert int to byte array([]byte) · GitHub
Golang : Convert int to byte array([]byte). GitHub Gist: instantly share code, notes, and snippets.
🌐
GitHub
github.com › arungudelli › go › blob › main › content › en › tips › string-to-byte-array-go.md
go/content/en/tips/string-to-byte-array-go.md at main · arungudelli/go
+++ title = "Convert a String To Byte Array or Slice in Go" date = "2020-12-13T14:09:21+09:00" description = "To convert a string to byte array or slice in Go language use []byte(string)" draft = "false" link = "convert string to byte array" ...
Author: arungudelli