First, a note on terminology: You appear to be talking about byte slices not byte arrays. Arrays in Go are of fixed length, so calling append on them won't work. See here for more.
Now to your question...
I am trying to make a for loop that will convert any data type to a byte array...
This is not possible, except in the most superficial, meaningless sense. For example, if you have a variable of type net.Conn, converting it to bytes will give you a meaningless value, since it is only meaningful in conjunction with one specific, active network connection.
However, assuming you don't have variables that refer to ephemeral state like this, the simplest way to convert arbitrary variables into a byte slice is with something like gob encoding.
However, this also has limitations, in that when serializing structs, it can only access exported fields, or those exposed through the GobEncoder interface. This Q&A explains why this limitation exists.
Now, in some cases you can work around this limitation with reflection, for example, but doing so is very cumbersome, and is usually unwise, since unexported fields often reference ephemeral state (see the note above).
So in summary: It's impossible to meaningfully do this for all data types. It's difficult to do it for many data types, and it's trivial to do it for "normal" data types.
Answer from Jonathan Hall on Stack OverflowStruct to byte[] vice versa
https://golang.org/pkg/encoding/gob/ this package is probably what you want mate :)
More on reddit.comgo - How to convert type to byte array golang - Stack Overflow
go - Convert []string to []byte - Stack Overflow
go - Convert arbitrary Golang interface to byte array - Stack Overflow
One possible solution is the "encoding/gob" standard package. The gob package creates an encoder/decoder that can encode any struct into an array of bytes and then decode that array back into a struct. There's a great post, here.
As others have pointed out, it's necessary to use a package like this because structs, by their nature, have unknown sizes and cannot be converted into arrays of bytes.
I've included some code and a play.
package main
import (
"bytes"
"encoding/gob"
"fmt"
"log"
)
type P struct {
X, Y, Z int
Name string
}
type Q struct {
X, Y *int32
Name string
}
func main() {
// Initialize the encoder and decoder. Normally enc and dec would be
// bound to network connections and the encoder and decoder would
// run in different processes.
var network bytes.Buffer // Stand-in for a network connection
enc := gob.NewEncoder(&network) // Will write to network.
dec := gob.NewDecoder(&network) // Will read from network.
// Encode (send) the value.
err := enc.Encode(P{3, 4, 5, "Pythagoras"})
if err != nil {
log.Fatal("encode error:", err)
}
// HERE ARE YOUR BYTES!!!!
fmt.Println(network.Bytes())
// Decode (receive) the value.
var q Q
err = dec.Decode(&q)
if err != nil {
log.Fatal("decode error:", err)
}
fmt.Printf("%q: {%d,%d}\n", q.Name, *q.X, *q.Y)
}
I assume you want something like the way C handles this. There is no built in way to do that. You will have to define your own serialization and deserialization to and from bytes for your struct. The binary package will help you encode the fields in your struct to bytes that you can add to the byte array but you will be responsible for specifying the lengths and offsets in the byte array that will hold the fields from your struct.
Your other options are to use one of the encoding packages: http://golang.org/pkg/encoding/ such as gob or json.
EDIT:
Since you want this for making a hash as you say in your comment the easisest thing to do is use []byte(fmt.Sprintf("%v", struct)) like so: http://play.golang.org/p/yY8mSdZ_kf
New to Go. Whats the best way to convert a struct to a byte[] and vice versa? I want to store the struct in cache.
Convert to json string then string to byte[]?
Thanks in advance!!!
Lets ignore the fact that this is Go for a second. The first thing you need is a serialization format to marshal the []string into.
There are many option here. You could build your own or use a library. I am going to assume you don't want to build your own and jump to serialization formats go supports.
In all examples, data is the []string and fp is the file you are reading/writing to. Errors are being ignored, check the returns of functions to handle errors.
Gob
Gob is a go only binary format. It should be relatively space efficient as the number of strings increases.
enc := gob.NewEncoder(fp)
enc.Encode(data)
Reading is also simple
var data []string
dec := gob.NewDecoder(fp)
dec.Decode(&data)
Gob is simple and to the point. However, the format is only readable with other Go code.
Json
Next is json. Json is a format used just about everywhere. This format is just as easy to use.
enc := json.NewEncoder(fp)
enc.Encode(data)
And for reading:
var data []string
dec := json.NewDecoder(fp)
dec.Decode(&data)
XML
XML is another common format. However, it has pretty high overhead and not as easy to use. While you could just do the same you did for gob and json, proper xml requires a root tag. In this case, we are using the root tag "Strings" and each string is wrapped in an "S" tag.
type Strings struct {
S []string
}
enc := xml.NewEncoder(fp)
enc.Encode(Strings{data})
var x Strings
dec := xml.NewDecoder(fp)
dec.Decode(&x)
data := x.S
CSV
CSV is different from the others. You have two options, use one record with n rows or n records with 1 row. The following example uses n records. It would be boring if I used one record. It would look too much like the others. CSV can ONLY hold strings.
enc := csv.NewWriter(fp)
for _, v := range data {
enc.Write([]string{v})
}
enc.Flush()
To read:
var err error
var data string
dec := csv.NewReader(fp)
for err == nil { // reading ends when an error is reached (perhaps io.EOF)
var s []string
s, err = dec.Read()
if len(s) > 0 {
data = append(data, s[0])
}
}
Which format you use is a matter of preference. There are many other possible encodings that I have not mentioned. For example, there is an external library called bencode. I don't personally like bencode, but it works. It is the same encoding used by bittorrent metadata files.
If you want to make your own encoding, encoding/binary is a good place to start. That would allow you to make the most compact file possible, but I hardly thing it is worth the effort.
The gob package will do this for you http://godoc.org/encoding/gob
Example to play with http://play.golang.org/p/e0FEZm-qiS
same source code is below.
package main
import (
"bytes"
"encoding/gob"
"fmt"
)
func main() {
// store to byte array
strs := []string{"foo", "bar"}
buf := &bytes.Buffer{}
gob.NewEncoder(buf).Encode(strs)
bs := buf.Bytes()
fmt.Printf("%q", bs)
// Decode it back
strs2 := []string{}
gob.NewDecoder(buf).Decode(&strs2)
fmt.Printf("%v", strs2)
}
Other problems in my code led me away from the gob package earlier, turns out it was the proper way as @nvcnvn suggested. Relevant code on how to solve this issue below:
package bloom
import (
"encoding/gob"
"bytes"
)
func GetBytes(key interface{}) ([]byte, error) {
var buf bytes.Buffer
enc := gob.NewEncoder(&buf)
err := enc.Encode(key)
if err != nil {
return nil, err
}
return buf.Bytes(), nil
}
Another way to convert interface{} to []bytes is to use a fmt package.
/*
* Convert variable `key` from interface{} to []byte
*/
byteKey := []byte(fmt.Sprintf("%v", key.(interface{})))
fmt.Sprintf converts interface value to string.
[]byte converts string value to byte.
※ Note ※ This method does not work if interface{} value is a pointer. Please find @PassKit's comment below.
Hi, I have an bitpack algorithm which needs a []byte slice e.g bitPack([]byte in, []byte out), my data is a float32 slice which I want to pack
Currently I'm doing the following:
func ToByte(in []float32) []byte {
return unsafe.Slice((*byte)(unsafe.Pointer(&in[0])), len(in)*4)
}
is this safe regarding GC, etc..., go vet at least does not complain.
Spec: Conversions does not allow to convert a byte to []byte. You may however create a slice that contains that single byte:
var value byte
data := []byte{value}
Also note that in your case to obtain a slice that contains the ith byte, simply use a slice expression:
characters[i:i+1]
Also don't forget that indexing a string indexes its UTF-8 byte sequence, and not the runes of the string, which does make a difference if the string contains runes that are encoded to multiple bytes (in UTF-8).
If you do want to iterate over the runes instead of the bytes, you could use a for .. range, or you could convert the input string to []rune first.
This seems to be what you're actually trying to do:
characters := "some text"
for i := 0; i < len(characters); i++ {
b := characters[i]
fmt.Println(b)
}
Your code is confusing (and doesn't compile), but you appear to want to iterate over the string, and for the element you are currently on, convert the character to a byte and print it. If so, you don't need (or want) to convert anything to a []byte.
If you don't want to iterate byte by byte but instead character by character, use a range loop. It iterates over runes:
characters := "søme spécial text"
for _, r := range characters {
char := string(r)
b := []byte(char)
fmt.Println(char, b)
}