Are ImageNet and Open Images geographically biased, and does it affect model accuracy outside the US and Europe?
Yes. In No Classification without Representation (NIPS 2017 workshop on ML for the developing world), we geolocated images in both datasets. About 45% of the ImageNet sample and over 32% of the Open Images sample came from the US, while China and India together held about 3%. Models trained on them scored bridegroom photos from Ethiopia and Pakistan less consistently than ones from the US and Australia.