RE: Space Track API - Problems

From: Max White (bmews@hotmail.com)
Date: Thu Aug 30 2012 - 06:57:58 UTC

  • Next message: Marco Langbroek: "Re: Space Track API - Problems"

    With all the comment about the new API system, how much is people feeding back into ST? It is OK to moan on this list but unless they get a barrage of noise that it is just not working and they have to reconsider their action of ditching the zip files, we will be stuck with what is on offer - which leaves serious gaps in the system.
     
    Ted, have you also invited their beta programmers to come onto this list to see the comments being made?
     
    73 Max
     
    
     
    
    > From: ciprian@sufitchi.com
    > To: seesat-l@satobs.org
    > Subject: RE: RE: Space Track API - Problems
    > Date: Thu, 30 Aug 2012 01:26:15 -0400
    > 
    > Simone,
    > 
    > You are right. I was able to retrieve the last 100,000 records ordered by
    > FILE, then I compared the result with the satellites found in the latest TLE
    > file from space-track.org. The difference is huge, many elements are
    > missing. Obviously by design the API is not supposed to generate a full TLE
    > file. Although it is possible to generate the file in multiple steps,
    > traversing and memorizing many catalog ids, programming skills are needed
    > and the process will be CPU intensive.
    > I think space-track is making a mistake with this new API. Currently users
    > are downloading once every few hours the TLE zipped file to cache the
    > elements for predictions. With this new API users are discouraged to cache
    > data and they will carry out frequent round trips to the API to retrieve the
    > elements for individual satellites. That would put a very high pressure on
    > the space-track system. Without caching user would have to rely on
    > space-track.org to respond fast. Unfortunately we've seen that
    > space-track.org is slow and unavailable quite often. With higher traffic
    > caused by API the situation will be worse. It would be great if space-track
    > would release this API for users, but keep the 2 and 3-element archived
    > files available for download.
    > 
    > Chip
    > 
    > -----Original Message-----
    > From: satrack@libero.it [mailto:satrack@libero.it] 
    > Sent: Wednesday, August 29, 2012 2:03 PM
    > To: ciprian@sufitchi.com; seesat-l@satobs.org
    > Subject: RE: RE: Space Track API - Problems
    > 
    > Hi Chip,
    > 
    > yes, this query corresponds to that I coded in my program, 
    > with the only difference that I request only the new files.
    > 
    > However, at the first run the result is identical. 
    > 
    > With this kind of solution I think we have to download much 
    > more than 25.000 TLEs to obtain all the elements.
    > 
    > Supposing the remote TLE table is named data_tle and that
    > it works properly with a such big DB, we would need a query
    > like this (maybe there is a more efficient one):
    > 
    > SELECT * FROM data_tle, (SELECT NORAD_CAT_ID, max(EPOCH) as lastepoch from 
    > data_tle group by NORAD_CAT_ID) as b where data_tle.EPOCH = b.lastepoch
    > 
    > but I think it is not implemented right now. 
    > 
    > Regards,
    > Simone
    > 
    > 
    > >----Messaggio originale----
    > >Da: ciprian@sufitchi.com
    > >Data: 28-ago-2012 23.20
    > >A: <seesat-l@satobs.org>
    > >Ogg: RE: Space Track API - Problems
    > >
    > >Simone,
    > >
    > >Did you try this query?
    > >
    > >https://beta.space-track.org/basicspacedata/query/class/tle/format/tle/orde
    > r
    > >by/FILE%20desc/limit/25000
    > >
    > >Chip
    > >
    > >-----Original Message-----
    > >From: seesat-l-bounces+ciprian=sufitchi.com@satobs.org
    > >[mailto:seesat-l-bounces+ciprian=sufitchi.com@satobs.org] On Behalf Of
    > >satrack@libero.it
    > >Sent: Monday, August 27, 2012 10:18 AM
    > >To: seesat-l@satobs.org
    > >Subject: Space Track API - Problems
    > >
    > >After a couple of days playing with the API, I found another
    > >problem of the available functions (the first one is the 
    > >missing 3 line format):
    > >
    > >If one wants to download the latest TLE for ALL the objects 
    > >(bulk catalog), two ways can be used:
    > >
    > >1) request the latest elements for all the objects using such
    > >a query:
    > >
    > >.../NORAD_CAT_ID/1--40000/sortby/EPOCH desc/sublimit/1
    > >
    > >unfurtunately this solution generates 40000 unions resulting
    > >in a query that is too long to work properly. 
    > >
    > >Try yourself and you will see that this method works only for
    > >a very limited range of NORAD_CAT_ID.
    > >
    > >I don't know if the situation will change later (after the test
    > >phase), but I fear this will not work.
    > >
    > >2) request a larger number of TLEs (let say 20.000) 
    > >starting from the new ones:
    > >
    > >.../FILE/>xxxxxx/oderby/FILE desc/sublimit/20000
    > >
    > >which is the solution abopted by my program.
    > >
    > >However also this solution doesn't work completely since:
    > >
    > >A) some elements are acquired only once in a few days 
    > >and they are not in the first 20.000 objects so they are 
    > >not downloaded!!!
    > >
    > >If you compare the bulk.txt file after the first download, more 
    > >than 3000 Objects are missing!!!! (Most of them are DEB)
    > >
    > >B) if we want all the elements we have to generate a traffic 
    > >much larger than actually required. (e.g. tents of MB against 
    > >0.8 MB of the old zip bulk file).
    > >
    > >(this method works well only to keep the bulk file up-to-date,
    > >e.g. once a day).
    > >
    > >If someone of you has got any idea on how to retrieve only 
    > >the last TLEs of each object, I will be glad to implement is 
    > >in the downloader.
    > >
    > >
    > >Best regards,
    > >Simone
    > >
    > >
    > >_______________________________________________
    > >Seesat-l mailing list
    > >http://mailman.satobs.org/mailman/listinfo/seesat-l
    > >
    > >_______________________________________________
    > >Seesat-l mailing list
    > >http://mailman.satobs.org/mailman/listinfo/seesat-l
    > >
    > 
    > 
    > _______________________________________________
    > Seesat-l mailing list
    > http://mailman.satobs.org/mailman/listinfo/seesat-l
     		 	   		  
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: http://mailman.satobs.org/mailman/private/seesat-l/attachments/20120830/33300c6a/attachment.html 
    _______________________________________________
    Seesat-l mailing list
    http://mailman.satobs.org/mailman/listinfo/seesat-l
    



    This archive was generated by hypermail 2b29 : Thu Aug 30 2012 - 06:58:43 UTC