Data inconsistency on decoding

[jtolio]

Hello!

With a relatively recent maxmind country-level DB (let me know if you want my exact copy), we have found at least one IP address that decodes to a different country code based on which de-serialization type is used. This is a bit concerning (we expect it has something to do with registered vs represented countries but we haven't nailed it down).

Here's the difference:

var data map[string]map[string]any
reader.Lookup("95.140.157.134", &data)
fmt.Println(data["country"]["iso_code"])

This says Russia.

var data struct {
	Country struct {
		IsoCode string `maxminddb:"iso_code"`
	} `maxminddb:"country"`
}
reader.Lookup("95.140.157.134", &data)
fmt.Println(data.Country.IsoCode)

This says Poland.

What's going on here?

[jtolio]

Additional information:

If I do the first style (map-based decoding), both "country" and "registered_country" say Russia.

However, if I do the second style (struct-based decoding), "country" is Poland, and "registered_country" is Russia.

So I think map-style is broken.

[oschwald]

I can't reproduce this:

package main

import (
	"fmt"
	"net"

	"github.com/oschwald/maxminddb-golang"
)

func main() {
	reader, err := maxminddb.Open("/usr/local/share/GeoIP/GeoLite2-City.mmdb")
	if err != nil {
		panic(err)
	}
	var data map[string]any
	err = reader.Lookup(net.ParseIP("95.140.157.134"), &data)
	if err != nil {
		panic(err)
	}
	fmt.Println(data["country"].(map[string]any)["iso_code"])

	var data2 struct {
		Country struct {
			IsoCode string `maxminddb:"iso_code"`
		} `maxminddb:"country"`
	}
	err = reader.Lookup(net.ParseIP("95.140.157.134"), &data2)
	if err != nil {
		panic(err)
	}
	fmt.Println(data2.Country.IsoCode)
}

Output:

PL
PL

I am guessing you are ignoring the error from Lookup, which might be related. I had to change the type to something that it could decode to.

[jtolio]

I only removed the error handling for the purposes of this issue submission. Here is a full test for me:

	type countryCode struct {
		Country struct {
			IsoCode string `maxminddb:"iso_code"`
		} `maxminddb:"country"`
	}

	var data countryCode
	err = mapper.reader.Lookup(net.ParseIP("95.140.157.134"), &data)
	if err != nil {
		panic(err)
	}
	fmt.Println(data.Country.IsoCode)

	var data2 map[string]map[string]any
	err = mapper.reader.Lookup(net.ParseIP("95.140.157.134"), &data2)
	if err != nil {
		panic(err)
	}
	fmt.Println(data2["country"]["iso_code"])

	var data3 map[string]any
	err = mapper.reader.Lookup(net.ParseIP("95.140.157.134"), &data3)
	if err != nil {
		panic(err)
	}
	fmt.Println(data3["country"].(map[string]any)["iso_code"])

This outputs:

PL
RU
PL

Notably, your formulation (map[string]any instead of map[string]map[string]any) works as expected.

[oschwald]

As suggested above, your version fails for me. I receive:

panic: maxminddb: cannot unmarshal array into type map[string]interface {}

goroutine 1 [running]:
main.main()
	/home/greg/MaxMind/maxminddb-golang/t/main.go:26 +0x2ba
exit status 2

This is because the subdivisions are an array, not map.

If I ignore the error, I see the behavior you describe. However, I wouldn't expect there to be valid data set by a method that returns an error and I wouldn't consider that a bug.

[jtolio]

This fails, even with the latest tag, v1.11.0? I didn't think to try the latest dev branch, but I will do that tomorrow. Perhaps there is some difference between your branch and the tagged release I am using. All of the returned errors are nil in my code--none of the panics in my example code above are triggered. Could this difference be in the format of the mmdb file I have?

[oschwald]

I have found an issue that I think is the cause of what you are seeing. I'll do a release shortly.

[oschwald]

1.12.0 has been released.

That said, I wouldn't recommend decoding to a map[string]map[string]any. It is much slower than decoding to a struct that contains just the data you want and it is more fragile than decoding to map[string]any as it makes possibly unfounded assumptions about what is in the database.

[jtolio]

Great, thank you! 1.12.0 is fixed. We do use the struct approach, but upon discovering this discrepancy we were worried which method was correct.