Downloading Flights¶
This notebook dumps flights to file using the OpenSky impala shell.
Ensure you have correctly configured the traffic library before use,
as well as setting the data location in
~/.config/flight_processing/flight_processing.conf.
from flight_processing import DataConfig
from flight_processing.data import FlightDownloader
import logging
logger = logging.getLogger()
logger.setLevel(level=logging.INFO)
logging.getLogger('traffic').disabled = True
Example 1: known dataset¶
This example uses the already encoded bounds of Switzerland to download flights.
First we download the flights as a Traffic object, then we dump 1
hour of flights to a file, then finally we dump 3 more hours of flights
to files in a single bulk command.
# Initialise downloader
downloader = FlightDownloader("switzerland")
# Download one hour of flights as a usable object
downloader.download_flights("2020-01-01 00:00", "2020-01-01 01:00")
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-01-01 00:00:00 and 2020-01-01 01:00:00 from OpenSky.
INFO:root:Sending request between time 2020-01-01 00:00:00+00:00 and 2020-01-01 01:00:00+00:00 and hour 2020-01-01 00:00:00+00:00 and 2020-01-01 01:00:00+00:00
INFO:paramiko.transport:Connected (version 2.0, client OpenSSH_7.6p1)
INFO:paramiko.transport:Authentication (password) successful!
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1577836800.0 and hour<1577840400.0 and time>=1577836800.0 and time<1577840400.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/c564afed75a09badb9ce574a5b0e280a
INFO:numexpr.utils:Note: NumExpr detected 32 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
INFO:numexpr.utils:NumExpr defaulting to 8 threads.
| count | ||
|---|---|---|
| icao24 | callsign | |
| 4b1806 | SWR | 3599 |
| 4b180b | SWR1327 | 3596 |
| 4b17fb | SWR193V | 2156 |
| 424352 | AFL2605 | 1591 |
| 3965ab | AFR470 | 1546 |
| 4b5c4d | FLORI513 | 1385 |
| 4b5c61 | FLORI539 | 1011 |
| 300621 | IBRXA | 580 |
| 4b1808 | SWR121E | 540 |
| 471ee2 | WZZ6192 | 470 |
# Download flights and dump as JSON
downloader.dump_flights("2020-01-01 00:00", "2020-01-01 01:00")
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-01-01 00:00:00 and 2020-01-01 01:00:00 from OpenSky.
INFO:root:Sending request between time 2020-01-01 00:00:00+00:00 and 2020-01-01 01:00:00+00:00 and hour 2020-01-01 00:00:00+00:00 and 2020-01-01 01:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1577836800.0 and hour<1577840400.0 and time>=1577836800.0 and time<1577840400.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/c564afed75a09badb9ce574a5b0e280a
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland/20200101/0000.json.
# Download 3 hours of flights and dump as JSON files
downloader.dump_flights_bulk("2020-01-01 01:00", "2020-01-01 04:00")
INFO:flight_processing.data.flight_downloader:Downloading flights in bulk between 2020-01-01 01:00:00 and 2020-01-01 04:00:00.
INFO:flight_processing.utils:Executing function 3 times between 2020-01-01 01:00:00 and 2020-01-01 04:00:00 with time delta 1:00:00.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-01-01 01:00:00 and 2020-01-01 02:00:00 from OpenSky.
INFO:root:Sending request between time 2020-01-01 01:00:00+00:00 and 2020-01-01 02:00:00+00:00 and hour 2020-01-01 01:00:00+00:00 and 2020-01-01 02:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1577840400.0 and hour<1577844000.0 and time>=1577840400.0 and time<1577844000.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/d64591914eb130bfebbbc8c9f09583d7
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland/20200101/0100.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-01-01 02:00:00 and 2020-01-01 03:00:00 from OpenSky.
INFO:root:Sending request between time 2020-01-01 02:00:00+00:00 and 2020-01-01 03:00:00+00:00 and hour 2020-01-01 02:00:00+00:00 and 2020-01-01 03:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1577844000.0 and hour<1577847600.0 and time>=1577844000.0 and time<1577847600.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/499b9720d5965dd1bcdf84191f90b274
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland/20200101/0200.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-01-01 03:00:00 and 2020-01-01 04:00:00 from OpenSky.
INFO:root:Sending request between time 2020-01-01 03:00:00+00:00 and 2020-01-01 04:00:00+00:00 and hour 2020-01-01 03:00:00+00:00 and 2020-01-01 04:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1577847600.0 and hour<1577851200.0 and time>=1577847600.0 and time<1577851200.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/f3e766e3c770358e6940973dc90241ff
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland/20200101/0300.json.
# Cleanup
del downloader
Example 2: new dataset¶
This example manually encodes the bounds of Switzerland to download flights in the same way as above - this can be done with any country or region.
First we download the flights as a Traffic object, then we dump 1
hour of flights to a file, then finally we dump 23 more hours of flights
to files in a single bulk command.
dataset = DataConfig("switzerland-custom", minlon=5.3, maxlon=10.7, minlat=45.5, maxlat=48, detail=6)
downloader = FlightDownloader(dataset)
# Download one hour of flights as a usable object
downloader.download_flights("2020-03-05 00:00", "2020-03-05 01:00")
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 00:00:00 and 2020-03-05 01:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 00:00:00+00:00 and 2020-03-05 01:00:00+00:00 and hour 2020-03-05 00:00:00+00:00 and 2020-03-05 01:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583366400.0 and hour<1583370000.0 and time>=1583366400.0 and time<1583370000.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/28d19cb664ef3c215b572a288db119ad
| count | ||
|---|---|---|
| icao24 | callsign | |
| 4b17fe | SWR | 3599 |
| 3e0a38 | BPO245 | 2761 |
| 4ca7f9 | ABR1624 | 2383 |
| 01d78d | ECHO2 | 1991 |
| 44014a | EJU9044 | 1990 |
| 424350 | AFL2605 | 1660 |
| 06a1e6 | QTR8111 | 1539 |
| 4caa86 | ANE2021 | 1400 |
| 451dbd | BCS130 | 622 |
| 4b43aa | RGA2 | 545 |
# Download flights and dump as JSON
downloader.dump_flights("2020-03-05 00:00", "2020-03-05 01:00")
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 00:00:00 and 2020-03-05 01:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 00:00:00+00:00 and 2020-03-05 01:00:00+00:00 and hour 2020-03-05 00:00:00+00:00 and 2020-03-05 01:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583366400.0 and hour<1583370000.0 and time>=1583366400.0 and time<1583370000.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/28d19cb664ef3c215b572a288db119ad
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/0000.json.
# Download 23 hours of flights and dump as JSON files
downloader.dump_flights_bulk("2020-03-05 01:00", "2020-03-06 00:00")
INFO:flight_processing.data.flight_downloader:Downloading flights in bulk between 2020-03-05 01:00:00 and 2020-03-06 00:00:00.
INFO:flight_processing.utils:Executing function 23 times between 2020-03-05 01:00:00 and 2020-03-06 00:00:00 with time delta 1:00:00.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 01:00:00 and 2020-03-05 02:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 01:00:00+00:00 and 2020-03-05 02:00:00+00:00 and hour 2020-03-05 01:00:00+00:00 and 2020-03-05 02:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583370000.0 and hour<1583373600.0 and time>=1583370000.0 and time<1583373600.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/8e0925cfa17b75fa340f324dde14be5b
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/0100.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 02:00:00 and 2020-03-05 03:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 02:00:00+00:00 and 2020-03-05 03:00:00+00:00 and hour 2020-03-05 02:00:00+00:00 and 2020-03-05 03:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583373600.0 and hour<1583377200.0 and time>=1583373600.0 and time<1583377200.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/768917fa05acf32f1b0afb2d9e8d346f
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/0200.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 03:00:00 and 2020-03-05 04:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 03:00:00+00:00 and 2020-03-05 04:00:00+00:00 and hour 2020-03-05 03:00:00+00:00 and 2020-03-05 04:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583377200.0 and hour<1583380800.0 and time>=1583377200.0 and time<1583380800.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/aed113955f3719266442fb2043f6fc63
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/0300.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 04:00:00 and 2020-03-05 05:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 04:00:00+00:00 and 2020-03-05 05:00:00+00:00 and hour 2020-03-05 04:00:00+00:00 and 2020-03-05 05:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583380800.0 and hour<1583384400.0 and time>=1583380800.0 and time<1583384400.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/99ec6c6a35b95a313b6037c93e0c7c16
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/0400.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 05:00:00 and 2020-03-05 06:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 05:00:00+00:00 and 2020-03-05 06:00:00+00:00 and hour 2020-03-05 05:00:00+00:00 and 2020-03-05 06:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583384400.0 and hour<1583388000.0 and time>=1583384400.0 and time<1583388000.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/defdd4e424d43bcdeffda3431fcc0310
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/0500.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 06:00:00 and 2020-03-05 07:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 06:00:00+00:00 and 2020-03-05 07:00:00+00:00 and hour 2020-03-05 06:00:00+00:00 and 2020-03-05 07:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583388000.0 and hour<1583391600.0 and time>=1583388000.0 and time<1583391600.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/7657b84539c72f6f617fa4f5095a1f58
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/0600.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 07:00:00 and 2020-03-05 08:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 07:00:00+00:00 and 2020-03-05 08:00:00+00:00 and hour 2020-03-05 07:00:00+00:00 and 2020-03-05 08:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583391600.0 and hour<1583395200.0 and time>=1583391600.0 and time<1583395200.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/1b393565cff9a9d899b9fb460985bd31
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/0700.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 08:00:00 and 2020-03-05 09:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 08:00:00+00:00 and 2020-03-05 09:00:00+00:00 and hour 2020-03-05 08:00:00+00:00 and 2020-03-05 09:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583395200.0 and hour<1583398800.0 and time>=1583395200.0 and time<1583398800.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/e7baff3622fe47dde6ce7d1ddaf4fd2a
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/0800.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 09:00:00 and 2020-03-05 10:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 09:00:00+00:00 and 2020-03-05 10:00:00+00:00 and hour 2020-03-05 09:00:00+00:00 and 2020-03-05 10:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583398800.0 and hour<1583402400.0 and time>=1583398800.0 and time<1583402400.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/e64ea6e6ed69db83bda3c5cca3f6c8d8
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/0900.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 10:00:00 and 2020-03-05 11:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 10:00:00+00:00 and 2020-03-05 11:00:00+00:00 and hour 2020-03-05 10:00:00+00:00 and 2020-03-05 11:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583402400.0 and hour<1583406000.0 and time>=1583402400.0 and time<1583406000.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/6ed2a01f81890dca2fdb6aedc1acd0e7
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/1000.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 11:00:00 and 2020-03-05 12:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 11:00:00+00:00 and 2020-03-05 12:00:00+00:00 and hour 2020-03-05 11:00:00+00:00 and 2020-03-05 12:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583406000.0 and hour<1583409600.0 and time>=1583406000.0 and time<1583409600.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/3a94a12f2d398a9c3e44d1d35ae7b3e6
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/1100.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 12:00:00 and 2020-03-05 13:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 12:00:00+00:00 and 2020-03-05 13:00:00+00:00 and hour 2020-03-05 12:00:00+00:00 and 2020-03-05 13:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583409600.0 and hour<1583413200.0 and time>=1583409600.0 and time<1583413200.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/1a7860026859e0a77453a5d0364237c8
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/1200.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 13:00:00 and 2020-03-05 14:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 13:00:00+00:00 and 2020-03-05 14:00:00+00:00 and hour 2020-03-05 13:00:00+00:00 and 2020-03-05 14:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583413200.0 and hour<1583416800.0 and time>=1583413200.0 and time<1583416800.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/d5cdf044b798d884e72d3e33d67ea7e0
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/1300.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 14:00:00 and 2020-03-05 15:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 14:00:00+00:00 and 2020-03-05 15:00:00+00:00 and hour 2020-03-05 14:00:00+00:00 and 2020-03-05 15:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583416800.0 and hour<1583420400.0 and time>=1583416800.0 and time<1583420400.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/156af8f9396f40c58016f78d14cb2def
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/1400.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 15:00:00 and 2020-03-05 16:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 15:00:00+00:00 and 2020-03-05 16:00:00+00:00 and hour 2020-03-05 15:00:00+00:00 and 2020-03-05 16:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583420400.0 and hour<1583424000.0 and time>=1583420400.0 and time<1583424000.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/ecb8c2e81440276aabe8f4832a805c75
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/1500.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 16:00:00 and 2020-03-05 17:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 16:00:00+00:00 and 2020-03-05 17:00:00+00:00 and hour 2020-03-05 16:00:00+00:00 and 2020-03-05 17:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583424000.0 and hour<1583427600.0 and time>=1583424000.0 and time<1583427600.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/c45730969ec9b83ff33c4d3eff3dc60e
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/1600.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 17:00:00 and 2020-03-05 18:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 17:00:00+00:00 and 2020-03-05 18:00:00+00:00 and hour 2020-03-05 17:00:00+00:00 and 2020-03-05 18:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583427600.0 and hour<1583431200.0 and time>=1583427600.0 and time<1583431200.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/53ed8b22ff6b3044dfb091efffc627fc
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/1700.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 18:00:00 and 2020-03-05 19:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 18:00:00+00:00 and 2020-03-05 19:00:00+00:00 and hour 2020-03-05 18:00:00+00:00 and 2020-03-05 19:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583431200.0 and hour<1583434800.0 and time>=1583431200.0 and time<1583434800.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/952e19914edd6e8664d84df765281b9b
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/1800.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 19:00:00 and 2020-03-05 20:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 19:00:00+00:00 and 2020-03-05 20:00:00+00:00 and hour 2020-03-05 19:00:00+00:00 and 2020-03-05 20:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583434800.0 and hour<1583438400.0 and time>=1583434800.0 and time<1583438400.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/af571277f703242c929300e69b48aec8
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/1900.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 20:00:00 and 2020-03-05 21:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 20:00:00+00:00 and 2020-03-05 21:00:00+00:00 and hour 2020-03-05 20:00:00+00:00 and 2020-03-05 21:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583438400.0 and hour<1583442000.0 and time>=1583438400.0 and time<1583442000.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/ad27d798c072d7639bb8dd22fd33945d
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/2000.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 21:00:00 and 2020-03-05 22:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 21:00:00+00:00 and 2020-03-05 22:00:00+00:00 and hour 2020-03-05 21:00:00+00:00 and 2020-03-05 22:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583442000.0 and hour<1583445600.0 and time>=1583442000.0 and time<1583445600.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/167c4b60aa70a23e9c114ca47df3cd12
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/2100.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 22:00:00 and 2020-03-05 23:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 22:00:00+00:00 and 2020-03-05 23:00:00+00:00 and hour 2020-03-05 22:00:00+00:00 and 2020-03-05 23:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583445600.0 and hour<1583449200.0 and time>=1583445600.0 and time<1583449200.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/3984469e45cd2eae9fec9e43f49b7865
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/2200.json.
INFO:flight_processing.data.flight_downloader:Downloading flights between 2020-03-05 23:00:00 and 2020-03-06 00:00:00 from OpenSky.
INFO:root:Sending request between time 2020-03-05 23:00:00+00:00 and 2020-03-06 00:00:00+00:00 and hour 2020-03-05 23:00:00+00:00 and 2020-03-06 00:00:00+00:00
INFO:root:Sending request: select time, icao24, lat, lon, velocity, heading, vertrate, callsign, onground, alert, spi, squawk, baroaltitude, geoaltitude, lastposupdate, lastcontact, hour from state_vectors_data4 where hour>=1583449200.0 and hour<1583452800.0 and time>=1583449200.0 and time<1583452800.0 and lon>=5.3 and lon<=10.7 and lat>=45.5 and lat<=48
INFO:root:Reading request in cache /mnt/cold_data/josh/traffic_cache/opensky/2b540de75ee1f39ce61cba8b9f086b3c
INFO:flight_processing.data.flight_downloader:Converting flights to list of coordinates.
INFO:flight_processing.data.flight_downloader:Dumping coordinates to JSON string.
INFO:flight_processing.data.flight_downloader:Saving JSON flights to /mnt/cold_data/josh/processing/flights/switzerland-custom/20200305/2300.json.
# Cleanup
del downloader