According to http://golang.org/pkg/encoding/binary/#Read :
Data must be a pointer to a fixed-size value or a slice of fixed-size values.
So you can't use slice []byte in your structure. But you can use fixed size array for it.
Like this:
package main
import (
"bytes"
"encoding/binary"
"fmt"
)
type T struct {
A int16
B int8
C [256]byte
}
func main() {
// Create a struct and write it.
t := T{A: 99, B: 10}
buf := &bytes.Buffer{}
err := binary.Write(buf, binary.BigEndian, t)
if err != nil {
panic(err)
}
fmt.Println(buf)
// Read into an empty struct.
t = T{}
err = binary.Read(buf, binary.BigEndian, &t)
if err != nil {
panic(err)
}
fmt.Printf("%d %d", t.A, t.B)
}
Go playground
Answer from RoninDev on Stack OverflowAccording to http://golang.org/pkg/encoding/binary/#Read :
Data must be a pointer to a fixed-size value or a slice of fixed-size values.
So you can't use slice []byte in your structure. But you can use fixed size array for it.
Like this:
package main
import (
"bytes"
"encoding/binary"
"fmt"
)
type T struct {
A int16
B int8
C [256]byte
}
func main() {
// Create a struct and write it.
t := T{A: 99, B: 10}
buf := &bytes.Buffer{}
err := binary.Write(buf, binary.BigEndian, t)
if err != nil {
panic(err)
}
fmt.Println(buf)
// Read into an empty struct.
t = T{}
err = binary.Read(buf, binary.BigEndian, &t)
if err != nil {
panic(err)
}
fmt.Printf("%d %d", t.A, t.B)
}
Go playground
I think binpacker will be greate to handle this case:
package main
import (
"bytes"
"fmt"
"github.com/zhuangsirui/binpacker"
)
type T struct {
A uint16
B string
C []byte
}
func main() {
field1 := uint16(1)
field2 := "Hello World"
field3 := []byte("Hello World")
buffer := new(bytes.Buffer)
binpacker.NewPacker(buffer).
PushUint16(field1).
PushUint16(uint16(len(field2))).PushString(field2).
PushUint16(uint16(len(field3))).PushBytes(field3)
t := new(T)
unpacker := binpacker.NewUnpacker(buffer)
unpacker.FetchUint16(&t.A).StringWithUint16Perfix(&t.B).BytesWithUint16Perfix(&t.C)
fmt.Println(t)
}
In C, for a protocol stack, I'd have a byte array from the network. I'd inspect a couple of bytes to determine what the packet is supposed to be at least, and then cast it to the appropriate struct. Something like this:
bptr = <malloc'd buffer> *protocolStruct = *(ProtocolObject)bptr
In golang, do I use the binary package, the unsafe { }, or what?
Let''s assume I have a struct for an ARP request, something like:
type NativeAddress struct
AddressType uint32
Address [16]byte
}And my ARP request structure is
type ARPRequest struct {
SAP uint16
NSAP uint16
SrcAddress NativeAddres
DstAddress NativeAddress
RequestedAddress uint64
}How does this get encoded with something like Gob and how does it get received on the other end? Wasteful though it is, everything is aligned. Can I just assume it gets encoded linearly ion the wire? In ancient C, I could do a tagged union that worked the same way, but modern C doesn't do that, you really can't count on what two C compilers will generate for structs <-> bytes. Or am I making this harder than it has to be -- should I just encode everything, every packet in protobuf, avro etc? In JVM langues, I have the same problem so I was hoping Golang handled it better. Even with Gob, can two architectures using Golang encode and decode structs reliably and, I still have the problem, after I read the SAP and NSAP bytes, only then do I know what struct it actually is and thus, I can "cast" it, or does Gob take care of that? I can't use something like JSON encoding, this is binary date.
I'd love to just have something that read bytes and said "I just received struct X" or "I have no idea what I just got" if I can't do the C style tricks.
unsafe.Pointer is, well, unsafe, and you don't actually need it here. Use encoding/binary package instead:
// Create a struct and write it.
t := T{A: 0xEEFFEEFF, B: 3.14}
buf := &bytes.Buffer{}
err := binary.Write(buf, binary.BigEndian, t)
if err != nil {
panic(err)
}
fmt.Println(buf.Bytes())
// Read into an empty struct.
t = T{}
err = binary.Read(buf, binary.BigEndian, &t)
if err != nil {
panic(err)
}
fmt.Printf("%x %f", t.A, t.B)
Playground
As you can see, it handles sizes and endianness quite neatly.
I've had the same problem and I solved it by using the "encoding/binary" package. Here's an example:
package main
import (
"bytes"
"fmt"
"encoding/binary"
)
func main() {
p := fmt.Println
b := []byte{43, 1, 0}
myStruct := MyStruct{}
err := binary.Read(bytes.NewBuffer(b[:]), binary.BigEndian, &myStruct)
if err != nil {
panic(err)
}
p(myStruct)
}
type MyStruct struct {
Num uint8
Num2 uint16
}
Here's the running example: https://play.golang.org/p/Q3LjaAWDMh
So apparently im parsing a byte array which represent flags in macho file
How I can parse this into a struct.
type Flags struct {
noundefs bool
incrlink bool
dyldlink bool
bindatload bool
prebound bool
split_segs bool
lazy_init bool
twolevel bool
force_flat bool
nomultidefs bool
nofixprebindin bool
prebindable bool
allmodsbound bool
subsections_via_symbols bool
canonical bool
weak_defines bool
binds_to_weak bool
allow_stack_execution bool
root_safe bool
setuid_safe bool
no_reexported_dylibs bool
pie bool
dead_strippable_dylib bool
has_tlv_descriptors bool
no_heap_execution bool
app_extension_safe bool
nlist_outofsync_with_dyldinfo bool
sim_support bool
dylib_in_cache bool
}
func ParseIntoFlag(p *parser.Parser, data chan any) chan any {
buf := bytes.NewBuffer(p.Data[0:4])
fmt.Println(binary.LittleEndian.Uint32(p.Data[0:4]))
p.Data = p.Data[4:]
flags := Flags{}
binary.Read(buf, binary.LittleEndian, flags)
data <- flags
return data
}FYI: im new to golang and code may be not the best.
Hi. I figured out how to convert a Go struct right into byte slice without reflection or memory allocations:
package main
import (
"fmt"
"unsafe"
)
type Movement struct {
Opcode int
X float64
Y float64
Z float64
}
const (
movementSize = int(unsafe.Sizeof(Movement{}))
)
func main() {
mv := Movement{
Opcode: 32,
X: 13.34,
Y: 20.36,
Z: 45.13,
}
packet := (*[movementSize]byte)(unsafe.Pointer(&mv))[:]
fmt.Println(mv)
fmt.Println(len(packet))
fmt.Println(packet)
}The output is:
{32 13.34 20.36 45.13}
32
[32 0 0 0 0 0 0 0 174 71 225 122 20 174 42 64 92 143 194 245 40 92 52 64 113 61 10 215 163 144 70 64]
Now I would like to know how to convert a byte slice into struct in a similar manner. Yes, I know about the binary package, but that's not what I need.
You have two errors in your code. Firstly, your struct fields should be exported for encoding/binary to see it.
Secondly, [16]string means an array of 16 strings, not a 16-byte string. So your struct should look like this:
type login struct {
Seq uint8
ID uint16
Username [16]byte
Password [16]byte
Unknown1 [16]byte
}
Then it works: https://play.golang.org/p/Nq8fRqgkcp.
You can do that using the unsafe package.
type test struct {
a int
s string
}
v1 := test{
a: 5,
s: "sdaf",
}
fmt.Printf("v1: %#v\n", v1)
b := *(*[unsafe.Sizeof(v1)]byte)(unsafe.Pointer(&v1))
fmt.Printf("bytes: %#v\n", b)
v2 := *(*test)(unsafe.Pointer(&b))
fmt.Printf("v2: %#v\n", v2)
One possible solution is the "encoding/gob" standard package. The gob package creates an encoder/decoder that can encode any struct into an array of bytes and then decode that array back into a struct. There's a great post, here.
As others have pointed out, it's necessary to use a package like this because structs, by their nature, have unknown sizes and cannot be converted into arrays of bytes.
I've included some code and a play.
package main
import (
"bytes"
"encoding/gob"
"fmt"
"log"
)
type P struct {
X, Y, Z int
Name string
}
type Q struct {
X, Y *int32
Name string
}
func main() {
// Initialize the encoder and decoder. Normally enc and dec would be
// bound to network connections and the encoder and decoder would
// run in different processes.
var network bytes.Buffer // Stand-in for a network connection
enc := gob.NewEncoder(&network) // Will write to network.
dec := gob.NewDecoder(&network) // Will read from network.
// Encode (send) the value.
err := enc.Encode(P{3, 4, 5, "Pythagoras"})
if err != nil {
log.Fatal("encode error:", err)
}
// HERE ARE YOUR BYTES!!!!
fmt.Println(network.Bytes())
// Decode (receive) the value.
var q Q
err = dec.Decode(&q)
if err != nil {
log.Fatal("decode error:", err)
}
fmt.Printf("%q: {%d,%d}\n", q.Name, *q.X, *q.Y)
}
I assume you want something like the way C handles this. There is no built in way to do that. You will have to define your own serialization and deserialization to and from bytes for your struct. The binary package will help you encode the fields in your struct to bytes that you can add to the byte array but you will be responsible for specifying the lengths and offsets in the byte array that will hold the fields from your struct.
Your other options are to use one of the encoding packages: http://golang.org/pkg/encoding/ such as gob or json.
EDIT:
Since you want this for making a hash as you say in your comment the easisest thing to do is use []byte(fmt.Sprintf("%v", struct)) like so: http://play.golang.org/p/yY8mSdZ_kf
New to Go. Whats the best way to convert a struct to a byte[] and vice versa? I want to store the struct in cache.
Convert to json string then string to byte[]?
Thanks in advance!!!