Just when I thought I was turning the corner, another major setback; the house I live in has been having electrical problems, which started out with annoying voltage drops (lights flickering, device brownouts), and eventually graduated to blown-out circuit breakers. I’m not sure if the breaker panel is just that old or what but the necessary part had to be custom-ordered, so the house has been without full power for 4 days now. Only the essentials are up and running; unfortunately, my PC and server aren’t part of the deal.
Before that, however, as I was beginning to get things back in order, I was encountering further problems, again with my data provider. At their suggestion, I had switched back to CSV format as it turns out they had an end point that would let me download all the day’s pricing in one shot; or rather, with one call, but multiple shots. For some reason, that end point doesn’t give me all the prices, I’m not sure if it’s a timeout issue on their end, or if that’s just the nature of their system.
When I was downloading from them in JSON format, it would take me a couple hours to download everything, but it was everything, 40,000+ prices. That may very well be the issue now that I think about it, I’ve started my downloads at the same time, so perhaps within the 2 hours that it would take me to download their JSON data, the prices that the CSV data doesn’t have have been populated. This is quite frustrating considering I’m paying for their service; when I was on the free service (at another vendor), when 5pm rolled by, all their prices were set and ready to download in a single call (granted, I switched away from them because a lot of that info was faulty, but it was all there). That these people have inferior service on a paid membership is upsetting to say the least.
The electrician is supposed to be here first thing tomorrow to install the part and check a few things out, so hopefully tomorrow night I can get back to working on the program. I’ll have to add some code to recursively check my database to keep making that CSV call to fill in data as it becomes available; I’d like to find out just how long it takes these people to complete their price updates but whenever I send a support request that goes beyond anything like “this ticker is incorrect”, I get dead silence, so their support is unreliable.
I really hope I can get somewhere soon with this program, start making a little bit of cash so I can find another, more reliable data vendor (although there doesn’t seem to be too many at my current price range).
Before we powered down, I had also started on the very beginnings of what will hopefully lead to more predictive features I have in mind; the first one being creating indexes for various facets of the stock market. The first and most easy to set up being an index for the whole market. Yes, there are DJIA and other similar indexes, but the Dow only covers the 30 largest companies, it’s hardly an accurate look at the market as a whole. So if I create my own indexes to monitor and “trade”, I may be able to create some kind of signal that “overall” market conditions are favorable for buying or selling, and then act accordingly on individual stocks.