I may be the only one looking for this...

Talk about whatever

16 posts • Page 1 of 2
I wish I was more into python, then I probably wouldn't have to ask what is probably a stupid question...

How do you use kml_filter.py? Which version of python is it expecting? I get different errors depending on which I run it through. Also, what is it expecting as arguments?

The only reason I am going this route is because the useful Mymaps feature of Google maps craps the bed on files over a certain length, and I would love to be able to setup different layers for different (or lacking) encryption. The only reason for the multiple layers is that Maps doesn't seem to let you search for an encryption type out of all the plotted points, even though that data is clearly included in the ToolTip.

Any advice would be greatly appreciated.
hey tx,

Well, it was written and tested in 2.7, but I haven't checked it for 3.0 compat. It was written in an evening in response to feature requests from IRC, so it's super hacky, and feedback/improvements are welcome.
You'll need to add https://github.com/cleder/fastkml to your runtime environment - there are a few different ways of building project specific virtual environments in python, but I just use virtualenv.

Basic steps:
1. create a directory, enter that directory with the terminal prompt of your choice. Then, create a virtual environment:

Code: Select all

virtualenv wigleKmlTest

Code: Select all

source ./wigleKmlTest/bin/activate
(ymmv on slash direction and "source" command based on OS)

2. still in that directory, install fastkml into that environment

Code: Select all

pip install git+git://github.com/cleder/fastkml
3. download the script into that directory. Run it with the "-h" option for help on usage (again, unix slashes presented for illustration):

Code: Select all

python ./kml_filter.py -h
Thanks for the reply.

I was able to pip the fastkml bits.

I got it working, for some reason, I thought it would segment the files by encryption (to seperate files). Now I see that isn't the case. But, it does make them much easier to chop up since it is sorted now.

I had planned on spinning up some python skills, and I imagine it wouldn't be too hard to figure out how to get this to dump individual files based on encryption (or lack thereof). That would make it much easier to setup layers on Google Maps.

Thanks again.
I seems to work well with one file, but it keeps dying upon moving to the second file. I tried running it in every version of python I have..
Attachments
Capture2.PNG
Capture2.PNG (86.14 KiB) Viewed 27357 times
Capture.PNG
Capture.PNG (248.78 KiB) Viewed 27357 times
how embarrassing; there was a dumb copy-pasta mistake on line 132 - i've updated our hosted copy, or you fix the conditional yourself in your local copy:

Code: Select all

if folder.name == "WPA3:":

Code: Select all

wpa3 = folder
(not wpa2)
Thank you.

Since it keeps referencing folders, does that mean it was intended to split the files by encryption? Or is "folders" python syntax of some kind?
they're organizing nodes inside the KML - holders for actual feature nodes!
Got it. Thank you.

Thanks again for the fix.
Worked great.

Ended up with a pretty big text file, but at least it will be much easier to cut up.

Thanks.
I don't have time to do it myself (not until the weekend at least), but try:

1. adding an options flag for independent files
2. if the flag is set, instantiate file "per folder" (a list of "newdoc"s instead of one) and add the folder to each
3. make a list of outputnames with the folder type (e.g. options.outputname+type+".kml"
4. iterate through the docs and output names, writing to each
Trying to cut up this (500K line) kml file is turning into a nightmare. There must be some nuance to the structure that I keep throwing out of balance. No matter how I edit this thing, it errors out on import.

Instead of going this route, I know the Wigle app can filter the display, but can it be made so that that filter applies to the upload file? For instance, if I have it filter for WEP and Unencrypted, can it be set so that that is all the data that is actually saved? Or does that only apply to the display?

Thanks.
uhg, sorry - that's been sitting toward the bottom of our "TODO" list for a long time, mostly since there isn't much room for UI nuance in the very-compact layout of the uploads page.

I'll take a swag at the python script this weekend, since that seems like the easier of the two challenges. I'm guessing that "hiding" the folders you don't want to see doesn't help with the KML file?

-a
If I am the only person trying to do things this way, then don't worry about it. I appreciate the offer, but I can always just fine tune my scans in Kismet if I need to, Wigle is just so much more efficient.

As far as hiding the folders, I tried just doing simple cut and pastes on the kml file. For instance, grab all the WEP networks and paste them in a new file, but everytime I try that (no matter how many lines), I can't seem to re-import them anywhere. I added the KML header, along with the tags at header and footer, but there must be a spacing issue that I throw out of wack when I do that. I have now got the import to work all the way until the last line then it barfs on something.
Found a decent free tool which will take the Wigle KML files and dump a CSV which I can then filter based on encryption, but then Maps barfs on the too much data in a field issue.

I think for my serious mapping I just have to use Kismet. I really don't know what else to try here, kind of thought filter these files would be a simple thing, but apparently it is not.
I'm not surprised that mapping packages can't handle it - we have made a lot of trade-offs trying to make WiGLE's visualization work "well enough" for everyone everywhere.

We have new filtering/color schemes for the maps under development - is there something we could do there that would help you? (onlymine + filter by encryption?)

Cheers,

-ark

16 posts • Page 1 of 2

Return to “General Grabbag”

Who is online

Users browsing this forum: No registered users and 39 guests