Page 1 of 1

It's time to archive/expire old data

Posted: Tue Mar 22, 2016 7:16 am
by SilverbNet
In general, even the most conservative, tech-averse household will have their access point replaced every five years or so, usually by the ISP. More tech-friendly households will update every 2-3 years, yet the Wigle map puts everything from today to 15 years ago on equal footing.

I just did a survey of my entire neighborhood, which turned out great, and visually compared my findings with the map limited to 2001-2012; I couldn't find a single SSID in common. Almost everything from then is just gone. Yet one of the annoying things is that quite a few of these old access points are shown in green, as high-quality data, some of which haven't been seen since 2005. (I found a grand total of one "linksys" and two "NETGEAR" compared to the dozens of each shown before.) Aside from simply expiring, it seems like access points that haven't been seen in years should fade to red, and then fade out entirely, if they aren't seen anymore. At the very least, restrict the date slider to begin last year or the year before, so that only current data is shown by default, leaving historical data available for perusal.

Obviously, performance isn't a problem; the site is amazingly fast even with all historical data shown. This is really more of a usability thing.

Re: It's time to archive/expire old data

Posted: Sat Mar 26, 2016 10:58 pm
by bobzilla
As you say, the date slider is there to adjust to the year(s) of your choice. By default we show everything, which seems ok.

Re: It's time to archive/expire old data

Posted: Sun Apr 03, 2016 2:46 pm
by KFy5xHA3
As you say, the date slider is there to adjust to the year(s) of your choice. By default we show everything, which seems ok.
The slider is fine, but it would be nice if the settings could be stored in bookmarks. I can set maplat, maplon and mapzoom that way, but startTransID and endTransID are always reset to 20010000-00000 and 20170000-00000.

Re: It's time to archive/expire old data

Posted: Sun Apr 03, 2016 5:09 pm
by bobzilla
Hrm, we just fixed something similar, we'll look into that.

Re: It's time to archive/expire old data

Posted: Tue Apr 12, 2016 6:28 pm
by 32894156
There is a feature which would be useful to add.
Currently, if I set the time slider to 2016-2017, it shows on the map only the APs which where NEWLY OBSERVED during the 2016 time frame. How about adding a slider to show all APs which were OBSERVED during a given time range.
Explanation: Suppose an AP was first observed in 2010, and last observed in 2015. Setting this new slider to 2015-2016 would show that AP; currently setting the old slider to 2015-2016 would not show it.
This would be more useful to show the current known state of the world, than the existing time slider.

Re: It's time to archive/expire old data

Posted: Thu Apr 14, 2016 4:56 am
by bobzilla
Maps javascript now handles startTransID and endTransID from the url. Being able to filter on last time seen would take a bit more work to plumb through.

Re: It's time to archive/expire old data

Posted: Tue May 17, 2016 5:56 pm
by KFy5xHA3
Maps javascript now handles startTransID and endTransID from the url. Being able to filter on last time seen would take a bit more work to plumb through.
Thank you. The TransID changes look fine to me.

Having more filter options is always nice and the 'last time observed' idea seems useful, but I can live with the current options. Here are a few more crazy ideas if you decide to look into the 'last time observed' filter and if you like to create extremely slow queries ;-)
  • Only observed by me
  • Only observed by others
  • Observed by both me and other(s) (ignore anonymous?)
  • Observed by more than one user (ignore anonymous?)

Re: It's time to archive/expire old data

Posted: Fri May 27, 2016 2:28 am
by dattransit
To add to the above post, a filter that would allow FirstObservedByMe=Yes (onlymine) and a new filter ObservedByOthers=Yes/No may be interesting.