One thing I was pondering, your prompt was "If this location is over land, say 'Land'. If this location is over water, say 'Water'. Do not say anything else. x° S, y° W".
How precise were the coordinates, and could we verify on Google maps those exact points?
Reason I ask is the "map" has a coarse resolution. Each pixel covers a (variable size due to the projection) area. On a "real" map there would be an aggregation function over the area determining whether to represent the area as "land" or "water".
However, reducing it to a single point in input space is a different question. Suppose a very accurate model given a location in Phoenix Arizona, if at that very point their happens to be a swimming pool, and that model answers "water", is it right or wrong?
Now how to improve the question is a different challenge. Do we accurately give the reference boundaries of the shape covered by that pixel on earth? Do we give the center coordinate with a range trying to match the shape with a best fitting circle? Do we sample the shape with different point locations within that and aggregate over those? (The latter would be intuitively my choice as my hunch on how info regarding location coordinates would be in the model data would be more amenable to that than the former choices)
Anyways, loved the work and the write-up!
Edit with thoughts from the shower:
Maybe another option is to let the model "semantically resolve" it.
e.g.:
"If I where in a plane over x° S, y° W, would I likely be flying over mostly landmass or mostly open water? you answer will be "Land" for mostly landmass, "Water" for mostly open water. Do not answer anything else."
You could optionally add the altitude of the plane for varying resolution.
Another option could be do make the task more explicit.
"If you would draw a very low resolution binary image of a world map, with a 1 representing landmass and a 0 representing open water, where each pixel represents an x square kilometer radius, what would be the value for the pixel containing x° S, y° W in Cartesian coordinates? Your answer should be a single "1" or "0", nothing else."
Very nice investigation. Thanks for sharing.
One thing I was pondering, your prompt was "If this location is over land, say 'Land'. If this location is over water, say 'Water'. Do not say anything else. x° S, y° W".
How precise were the coordinates, and could we verify on Google maps those exact points?
Reason I ask is the "map" has a coarse resolution. Each pixel covers a (variable size due to the projection) area. On a "real" map there would be an aggregation function over the area determining whether to represent the area as "land" or "water".
However, reducing it to a single point in input space is a different question. Suppose a very accurate model given a location in Phoenix Arizona, if at that very point their happens to be a swimming pool, and that model answers "water", is it right or wrong?
Now how to improve the question is a different challenge. Do we accurately give the reference boundaries of the shape covered by that pixel on earth? Do we give the center coordinate with a range trying to match the shape with a best fitting circle? Do we sample the shape with different point locations within that and aggregate over those? (The latter would be intuitively my choice as my hunch on how info regarding location coordinates would be in the model data would be more amenable to that than the former choices)
Anyways, loved the work and the write-up!
Edit with thoughts from the shower:
Maybe another option is to let the model "semantically resolve" it.
e.g.:
"If I where in a plane over x° S, y° W, would I likely be flying over mostly landmass or mostly open water? you answer will be "Land" for mostly landmass, "Water" for mostly open water. Do not answer anything else."
You could optionally add the altitude of the plane for varying resolution.
Another option could be do make the task more explicit.
"If you would draw a very low resolution binary image of a world map, with a 1 representing landmass and a 0 representing open water, where each pixel represents an x square kilometer radius, what would be the value for the pixel containing x° S, y° W in Cartesian coordinates? Your answer should be a single "1" or "0", nothing else."
Fascinating experiment!