I suggested a few weeks ago that we try to come up with a way to measure the accuracy of the Streetcar NextBus predictions (since they have been a source of some complaints reported in the press).
Turns out someone has already done it (at least for a small data sample). Rob Bertini of PSU forwarded this student project (PDF, 307K) by Hiu Ng from last year.
As I read his data, the most common result seems to be that Streetcar arrives about 1 minute after the predicted time. If you think about how you would want to bias prediction error in such a system, if definitely seems like you’re better off having people arrive a little early than risk having the car already departed by the time you tell them it’s going to arrive!
So I think this confirms my anecdotal experience, but I’d still love to find a way to collect a larger data set.
35 responses to “NextBus Accuracy”
Any train schedule show departure time, not arrival time, has always been that way….
Normal trains generally have no traffic to contend with (west coast Amtrak NOT included) and can therefore be on time. I’m going to guess that the main reason that the streetcar’s actual arrival times are off is because of auto traffic. I think the best way compensate for that is via statistical gathering. Are the actual arrival times stored somewhere? If so, the data per stop can be analyzed over time to determine the average arrival time per stop and present a better prediction.
The biggest source of variation is actually passenger loading.
But there is no system to collect/record actual arrivals. That’s what I’d love to figure out.
How do the arrival time sign boards get their data? Is it a two way communication path? If so, it seems like it would be fairly easy to install sensors at each stop to determine exactly when a car is present/not present. With a high sampling rate, the arrival and departure times could easily be acquired. The data could then be sent up to a collection point.
There are GPS sensors on every vehicle.
So theoretically, we could ask NextBus for a log of when vehicles were at stops, but since we’re trying to verify NextBus’ accuracy, it would be better to have data from an independent system.
Er, the main delay for the Amtrak Cascades is freight rail traffic. Far and away.
Ok Chris, in that case, perhaps the best way is to have an actual human ride the route and collect data manually? Considering the cost of installing some other system simply to verify the existing system, that seems to be the more practical example, no?
And just for the record, I have never had a problem with the accuracy but I always try to show up early. And I’m lucky that my main pickup point is near one of the turns, I can sometimes hear it screeching along and make a run for the next stop.
Um, doesn’t TriMet do this with the TransitTracker???????????
The biggest problem I have with TriMet is:
1. Busses that don’t transmit data, so you never know if the bus will be there or not.
2. Busses that are delayed by several minutes, and the system “drops” them, although they haven’t arrived yet.
3. System doesn’t work in snow (TriMet shuts the system down). TriMet needs to work on a solution that provides at least some information, such as if the bus is operating and where the next bus is (instead of how many minutes to the next bus).
Ok Chris, in that case, perhaps the best way is to have an actual human ride the route and collect data manually? Considering the cost of installing some other system simply to verify the existing system, that seems to be the more practical example, no?
That’s what the PSU student did. I’d just like to get a larger data set than is economical using human measuring devices.
Um, doesn’t TriMet do this with the TransitTracker??????????? … System doesn’t work in snow (TriMet shuts the system down).
Streetcar has its own separate system (Transit Tracker wasn’t up yet when we needed to deploy) but work is going on to better integrate them.
I’ve given TriMet the same feedback about Transit Tracker during snow and ice (multiple times). At the time the system is MOST valuable, it stops working. They understand, but don’t have a solution yet.
Erik –
I agree completely that Transit Tracker needs to be modified/improved so that rather than dropping buses that fit outside its parameters and rather than being shut down during snow, it should display some other kind of pertinent information.
At the very least, it should be possible to program the system to display locations of the last verified detection of a bus (closest to you along the route). For example, if you’re waiting for the #71 southbound at Glisan on an icy morning at 7:40am, knowing that the #71 was “Seen at Sandy Blvd, 7:35am” will help you decide whether or not to seek alternatives, or at least step inside somewhere for some hot coffee.
– Bob R.
What trimet needs to do is create wireless internet access points on each bus/train so that people can connect to the internet. Then have a device that sends the route number, bus and GPS coordinates at each time stop and have code on the web site translate that into information that can be accessed on the web or mobile phone. Then put wireless internet connections on or near the bus stops and use that for the message boards to pick up where the bus is.
In short, Chris, Trimet has been struggling for too long with what should be a relatively easy technical task (even if my solution is far-fetched). I am not sure it is really seen as a priority. I think part of it is the age of a lot of the senior leadership who still see this kind of technical innovation as geeky. Because of that you need to get some folks who are less geeky than you to make the case.
Ross, I think we sort of agree, although perhaps not on the technical solution.
TriMet’s data is available by cell phone (voice) today, indeed that is there primary strategy. TriMet also makes it available on the web, and cell phones with web access can also get it in a format tuned for small screens (although of course we think our Transit Surfer has a better interface).
I think you’re suggesting TriMet should make it available via WiFi, but I’m not sure that has much payback, as WiFi-enabled phones are still much rarer than phones with web browsers (that may change).
[TriMet still might want to put WiFi on the Red Line so they could advertise laptop web access from the Airport to downtown!]
My wish is that they would put display devices at all stops, so you don’t have to mess with a device, you just look up (as with Streetcar and Nextbus). TriMet has backed away from this (even removed some of the displays they used to have) because of the communications costs involved. I believe they are part of the agreement with MetroFi for Portland’s new municipal WiFi network, but I suspect they would still have a finite cost per display. They’re banking on people using their cell phones instead, but I think that misses a big chunk of the market.
I’ve actually been fooling around with one of the new Nokia 800 internet devices to see if it could be turned into such a display, but haven’t had as much time as I’d like to work on it :-)
The absolute LAST thing TriMet needs to do is install Wi-Fi.
TriMet is a TRANSIT agency. If TriMet can’t even get their damn busses on the road consistently and reliably, why should they WASTE TAXPAYER DOLLARS on something that isn’t even in their job description?
When TriMet can serve 100% of its transit district with frequent and reliable transit service (regardless of mode, even if it’s just a lowly crappy bus) – then let’s talk Wi-Fi access. Until then, TriMet needs to stick to what is in its legally defined mission.
If TriMet is going to offer free Wi-Fi, then TriMet might as well offer free laptop computers to every man, woman and child in its service district, too.
TriMet has backed away from this (even removed some of the displays they used to have) because of the communications costs involved.
That’s because TriMet made a lousy decision to use a cellular network based system, instead of using a data radio that could be cheaply deployed. There are already public service data radio networks blanketing the metro region that don’t have monthly operating costs, other than maintenance costs and the annual FCC license fee. But TriMet, in their infinite wisdom (because we all know that TriMet knows best and TriMet is perfect at everything), chose something that required a cell phone for every site off of the MAX line (where it uses the MAX communications backbone).
As a result, precious taxpayer dollars on what seemed like a decent service, is sitting in some warehouse as a prime example of government waste. Did anyone get fired for that?
Erik, I gotta say that your response makes me think of the difference in glass half full/half empty thinking.
My perspective is TriMet is ahead of pretty much any other transit agency in the U.S. on deploying this technology and we need to help them steer down the right course.
Your perspective is they made a wrong choice so we should fire somebody.
I hope they keep taking (well-calculated) risks on innovative technology.
Meanwhile we need help educate them about why displays you can view passively will make much more impact on ridership than approaches that require active effort like cell phones or web browsers.
Your perspective is they made a wrong choice so we should fire somebody.
Chris,
If TriMet had been reasonable, it would have started a pilot project at one or two transit centers to see how the system works.
It would have caught the problem early on, developed a better way to do it, and then do a wide-spread deployment.
Instead, TriMet developed something, deployed it, found a bug that couldn’t be fixed, and scrapped it at a huge cost.
There’s a reason why large companies don’t invest huge sums of money in a new operating system as soon as it’s released (how many companies run Windows Vista?) – they want until the bugs are worked out, and probably install Vista on a few PCs in the I.T. room.
The same with TriMet’s experiment in Hybrid busses (of which TriMet has two). I congratulate TriMet on doing this (at the time).
The problem is that TriMet hasn’t found anything wrong with these two busses, and they still only have two. Other transit agencies that actually invest in bus service (which TriMet doesn’t) have sprung so far ahead of TriMet in deploying what is now a proven technology that TriMet can’t claim jack in being “environmentally conscious” towards its bus network. Even little Cherriots, down in Salem, which is financially strapped (not because of waste but because of a lack of funding mechanism) is ahead of TriMet by being almost 100% CNG powered.
(I do know that TriMet did have several CNG powered Flxible Metro busses, and TriMet passed up the technology. Those two busses were converted to diesel operation and the roof-mounted CNG tanks were removed at the same time. AFAIK, the jury is still out on CNG, because there are certain expenses involved (namely fueling facilities) where the cost doesn’t result in a savings over diesel.)
Back to transit trackers – If TriMet had kept the transit trackers alive and well at the transit centers (other than on MAX), then yes – TriMet would be ahead of most other transit agencies. Unfortunately TriMet has removed those systems – and took a step back.
Someone paid for that decision. Since Fred Hansen still gets his paycheck the same either way, it’s the taxpayers and riders (and in my case my employer) that foots the bill. BTW, why does it cost me 25% more to ride TriMet than it does to ride King County Metro (and that’s the Rush Hour fare, outside of Rush Hour it’s 38% more)?
Yes, someone should get fired. If my I.T. director had deployed Windows Vista on our PCs, and a major security breach or a OS bug occurred, that required every PC in our company to be rolled back, I can assure you that he’d be fired. I hold TriMet to the same standard; I see no reason to be an TriMet apologist or to accept a lower standard of quality for TriMet.
Portland Streetcar has the NextBus displays, and it works given that Streetcar is a very limited network and has a communications system that is effective and doesn’t have a per-unit cellular phone expense. But Streetcar isn’t TriMet…
Interestingly, NextBus uses a private radio network, according to their website.
So, why didn’t TriMet just pay NextBus to create the network?
NextBus uses cellular communication (Cingular) and we pay per location. We just made the call that it was worth it.
So how many displays did TriMet scrap? My impression is that it’s a relatively low number (but I don’t know) and that really the choice was whether or not to roll it out further and they chose not to.
If TriMet had been reasonable, it would have started a pilot project at one or two transit centers to see how the system works.
Which is, in fact, what they did.
At which transit centers was TransitTracker readerboards deployed to? (I know Barbur Blvd. TC was one.)
And, if the program was such a success (except for the communications costs), why didn’t TriMet come up with a better solution, instead of scrapping it?
For example:
1. Tying onto the City of Portland mobile data radio network (that TriMet’s own busses use, ironically),
2. Building a new TriMet radio network, that could also be used with the TVMs,
3. Building a landline phone network that would use existing infrastructure used for the payphones; could also be used for emergency intercom/communications system or CCTV access for transit centers…
There was one at 5th and Salmon downtown.
One TriMet employee told me the signs were “cobbled together in the shop” so I don’t think they spent bigs bucks on them.
And TriMet DOES have a strategy, it’s to get people to get this info by voice over cell phone. I don’t think the strategy is optimal, but you can’t say they didn’t have a deliberate strategy.
I can definitely see both sides of this issue. I have a cell phone, so I think it’s quite easy to use the existing Tri-Met system. My parents, on the other hand, hate cell phones and won’t own one, period. So, they’re rather at a loss with the Transit Tracker system, and pretty much can’t use it.
It would seem that reader boards should be installed at all shelters. Wi-Fi would seem to be the best available technology, since there is this WiFi cloud that is in theory spreading across Portland. Maybe each transit shelter could also become a repeater or a hub, depending on local infrastructure, in the WiFi cloud? Cost-sharing should probably take place with the City and with the WiFi provider — I agree that Tri-Met should spent all of its operating funds on operating buses and light rail, so unless they can receive some separate capital funding for installation of wifi/transit tracker… they need to collaborate.
Might this be a win-win for WiFi, Tri-Met and Tri-Met’s customers?
Wi-Fi would seem to be the best available technology, since there is this WiFi cloud that is in theory spreading across Portland
TriMet stands for Tri-County Metropolitan Transportation District of Oregon.
WiFi is not universal across the TriMet service area. Using a private data network, such as already is used by TriMet for its bus dispatch system, by local law enforcement for the mobile data terminals – is universal and has existing radio facilities to support it.
Even if a Transit Tracker v.2 needed new radios, at least the antenna arrays exist, the radio shacks exist – it’s just moving the equipment, and installing the readerboards.
WiFi would work in the downtown Portland area (I don’t see why Streetcar is still using GPRS when WiFi is available and free), but it wouldn’t work for TriMet, particularly in the outer areas of the district.
Streetcar is coming the end of its five year contract with NextBus soon. I expect that we’ll be re-evaluting a lot of elements of the program. 5 years ago Transit Tracker didn’t exist and cellular was the only option.
While I has a cell phone, and even have transit tracker on speed dial, a lot of bus stops don’t have their stop ID on the sign, (I fixed this problem at one of the ones near my house with a sharpie, and will probably be arrested for vandalism for telling you about it,) and while the IVR system is actually quite good as IVRs go, it still takes about 2 minutes to get a time out of it…
Maybe TriMet should have a plan to upgrade their bus stop signs or something like that. :-P
There seems to be a lot of work going on right now with Google Transit to open up a specification for providers to share schedule data with developers. Our own TriMet is one of two agencies (along with BART) to open up an actual public feed:
http://code.google.com/p/googletransitdatafeed/wiki/PublicFeeds
Instead of being limited to what TriMet’s developers can come up with, this data is open and available to anyone who can come up with a system to display it usefully in a variety of contexts. Why limit yourself to WiFi, cellular or private networks when you could use any/all/none of them?
This is schedule data, not real-time. However it isn’t too far-fetched to believe that someday Transit Tracker data may be available through some sort of API. All sorts of possibilites are out there when you open it up.
TriMet’s real-time data is already available via a SOAP interface, and our own Transit Surfer makes use of it.
First of all, TriMet has been upgrading bus stop signs (to half-moon shapes), and many of these now have schedules, *including* the Stop ID.
As for the bus stop Transit Tracker displays, it sounds like they may have had problems switching from the cellular connections. See …TransitTrackerSelfEvaluation.pdf or search Google for: “transit tracker” filetype:pdf
And there were plenty of displays: …onstreetlocations.htm
why didn’t TriMet just pay NextBus to create the network?
I’m pretty sure that they didn’t want something proprietary, or that it wasn’t an option when they started.
As for the hybrid buses, Fred Hansen says: “However, the technology has not advanced as quickly as we would have liked, and we are not achieving the fuel savings we were hoping for. We are in the process of analyzing our experience with the bus and will produce a white paper on our findings in the next few months.”
Lastly, another improvement to Transit Tracker would be compensation for buses that haven’t moved.
[Editor’s note: Links edited to restore page formatting. BR – 2007-05-20]
Yeah, I totally support the program to upgrade the bus stops. It is a very cheap/easy way to increase ridership with a much higher return on investment than say, buying more buses. The VTPI thread a couple weeks ago had a lot of good data on that, (although given the discussion, I don’t think everyone read the entire pdf.)
The bus schedule on the signs is a good idea, but it unfortunately has been poorly implemented: A lot of the schedules are actually a year or so old, which for most routes means that the times are completely wrong, (moving the bus mall didn’t help, but that wasn’t the only cause.) While the VTPI study didn’t go into the value of incorrect schedules, I’m fairly sure that it is worse than having no schedule posted at all. The problem is that TriMet changes at least some of the schedules every few months, so they’d pretty much have to employ a few people to just walk each bus route with a cordless drill and a new set of schedules and more or less continuously replace the ones in the signs… And that would be expensive. I wonder if printing them up as waterproof stickers, and so they could just slap them over the top of the old one without having to open the boxes would be cheaper?
TriMet has been upgrading bus stop signs (to half-moon shapes), and many of these now have schedules, *including* the Stop ID.
WOW!!!!! Those half-moon bus stop signs make me want to ride the bus!!!!!
That’s absurd, to say the least. Those signs were downright a WASTE OF MONEY – especially on the westside, where TriMet had installed hundreds of brand new, plain ordinary rectangular blue and white signs in the years before. They do nothing for schedule reliability or comfort on the bus, or making the transit experience better.
By the way, why does my bus stop sign still have a schedule for the 95? And in November TriMet had posted a Rider Alert stating that my “BUT STOP MOVING” (yes, TriMet actually mis-spelled “Bus” and when the sign fell off I preserved it for history. It was the laughingstock of my workplace.) Yet the bus stop was never moved; in fact TriMet did take the sign down, covered up the ’95’ on the sign, put a new sign base, and put the sign back in the exact same non-ADA compliant location.
However, the technology has not advanced as quickly as we would have liked, and we are not achieving the fuel savings we were hoping for.
Then why are bus agencies like Coast Mountain Bus and King County Metro going to hybrid busses? I think something is fishy…
The bus schedule on the signs is a good idea, but it unfortunately has been poorly implemented: A lot of the schedules are actually a year or so old, which for most routes means that the times are completely wrong
You mean, like a bus stop listing schedules for the 95? A bus line that TriMet discontinued?
As for the bus stop numbers, if you don’t have a cell phone with WAP access (or a laptop computer, and depending on your location either Wi-Fi or cellular data access), those numbers are useless. They don’t mean anything.
Erik said:
Well, I agree that, while Wi-Fi might work in Portland, it won’t work where it doesn’t exist. Not sure this is an argument against using WiFi, or an argument for growing the cloud, however. I just don’t know enough about how well it might or might not work, in comparison with the other technologies being discussed… I just wanted to bring it up for discussion as an option. Again, it seems like using bus shelters as repeater/hub locations could bring WiFi service, both to the transit system and to the surrounding neighborhood.
As for the bus stop numbers, you can just call 503-238-RIDE to use them to get your arrival times. No need to login to a website using WAP. The regular telephone interface works just fine — I’ve tried it.
As for the bus stop numbers, you can just call 503-238-RIDE to use them to get your arrival times. No need to login to a website using WAP. The regular telephone interface works just fine — I’ve tried it.
Agreed, but you still need a cell phone (if you’re standing at a bus stop) to use it (so no need for WAP). I prefer WAP because it’s quicker, and I can refresh it (or if I’m at a busy bus stop I’ll let others know when their busses are scheduled to arrive).
Again, it seems like using bus shelters as repeater/hub locations could bring WiFi service, both to the transit system and to the surrounding neighborhood.
I’m NOT opposed to using bus shelters as locations for WiFi repeaters.
I’m opposed to TriMet using dollars for it that should be used to improve/enhance transit service, as well as maintaining its current infrastructure (i.e. maintain/replace current vehicle fleet).
If a private company (i.e. MetroFi) wants to install WiFi repeaters on bus shelters and market such; while giving TriMet the ability to provide arrival times using the same network, I’m certainly for that. However, TriMet also needs to have a solution available so that those “in the boonies” aren’t left out – because TriMet has an obligation to serve everyone equally, regardless of whether you’re in the Pearl, Rockwood, or Forest Grove.
If, on the other hand, TriMet uses $300K to provide Wi-Fi, and turns around and says they have to reduce service on X route, or that it has to cancel a planned purchase of new busses due to funding, that IS a problem. TriMet’s priorities are to provide transit, not internet access.
Agreed. Service should be the first priority.
But perhaps a private company could be hired to make the whole wifi thing happen? They could put an ad up at each shelter, and use the ad revenue to fund the reader board as well as the wifi infrastructure.
Of course, that would mean accepting advertising at each shelter in the community.
But it’s already on the side of the bus. What’s the difference? Especially if it pays for the reader board and the technology to make it work?
Tri-Met could solicit for a system-wide shelter/advertising contract, negotiated in tandem with each city in the service district, and stipulate that every shelter must provide the bus arrival time information, as well as serve as a wifi hub or repeater. The state of the art for these things is to make each shelter self-powering with wind/solar technology & batteries.
Is it worth it to accept the ads, when all of these benefits could also be provided?
The same vendor could, incidentally, potentially also serve the bicycle rental contract… though there’s no reason from the public’s perspective that they would need to be linked. A stylistic link might be aesthetically pleasing, however.
Chris –
I’d like to wake this thread back up… I have an idea that I think might provide the kind of answers that you want.
By using a stand-alone remote webcam, and by recruiting the help of friendly businesses/organizations with Internet access and office space which can view streetcar stops, a monitoring program can be created which, with a little human intervention at the end of the project, can generate some statistical answers.
The short version goes like this:
1. Place webcam to look at a particular stop for a few days.
2. Create a program which monitors the NextBus predictions, available on the PortlandStreetcar.org web site, and gathers a series of photographs from the remote web cam, storing them for future reference.
3. At the end of the data-gathering period, a human can briefly view the snapshots and click on the ones that best represent the arrival of a streetcar. This will require a about 200 “clicks” for a full day of service, but in real terms this should take well under an hour to compile.
The longer version goes like this…
Assumptions:
1. The further away a streetcar is, the more variance there will be in a prediction (NextBus predictions go up and down as the progress of a streetcar is measured.) We should pick a time window in which it is reasonable to expect that the prediction be accurate. For example, a prediction of 15 or 20 minutes may cause most people to move on or find something else to do rather than stand and wait, but a prediction of 10 minutes is short enough that most people will elect to stand and wait. Thus, it is important to start our tracking when a prediction first falls under 10 minutes, and then determine the final accuracy of that prediction.
2. Assume that a streetcar which is more than X minutes late is a failed prediction. While our test will measure just how accurate the predictions really are, we also need to have a cut-off where it is no longer important to gather data, say after 5 minutes late.
3. It is important that streetcars not depart too early… a person receiving a web-based tip that the streetcar near their office is due in 10 minutes may be upset, 5 minutes later when arriving at the stop, to find that the streetcar has departed. Thus, we should begin gathering image data as soon as time window in assumption #1 has been entered. We should further assume in our end results that a significantly early streetcar (say 3 or 4 minutes) is a failed prediction.
Implementation:
1. Place a remote webcam where a streetcar stop, preferably at least mid-way along the line from a layover point, can be seen. (For example, this webcam requires no PC at the site and is remotely adjustable for pan/tilt in case it gets nudged.)
2. Write a remote program (I’m not volunteering yet but I could probably throw something together on remote server running PHP, which I’m familiar with) which monitors the NextBus prediction for the stop, and when that prediction falls under 10 minutes, begin storing frames every 15 seconds or so, until either A) 5 minutes after the current prediction or B) a new prediction supersedes it and falls within the 10-minute window.
3. Each stored frame will be stamped with three values: The actual time (from time.gov), the NextBus time-of-day, and the current prediction. This should cover the bases for many kinds of analysis.
4. At the end of the data-gathering period, a user with a web browser can be presented with thumbnails of all the stored images and simply click on the photos containing an arrived, stopped streetcar to mark the real arrival time.
5. It is important that during the test period, dispatchers for Portland Streetcar maintain accurate logs of all service interruptions, delays, traffic tie-ups, etc. which are outside of the normal boundaries of what NextBus is expected to handle. These logs can be compared with the data set to determine if a failed prediction was caused by an external event.
I have noticed that the Central Library has rooms which provide an excellent view of the streetcar stops on either side of the building… perhaps this would be a good place to take samples. Another place might be the Shiels | Obletz | Johnsen offices.
Short of writing a software program to manage this, it may be sufficient to simply set up the webcam and schedule a few PortlandTransport volunteers to watch the thing, each for an hour, and email their observations to a point person who will compile the data.
– Bob R.
Bob,
I can certainly ask SOJ about putting a webcam in their office. It would probably need to be firewall-friendly.
Streetcar already keeps pretty good operations logs, but it might be simpler to just track when there is a service interruption or delay message going out over NextBus.
The software that pulls NextBus data for use in the Transit Surfer already has the ability to log the service messages.
Chris –
Several of the webcam models out there can be set to a static IP address — the volunteer organization “hosting” the webcam will need to update their router/firewall to expose/map just that one IP address to the outside world.
It will also be important to physically disable any audio capability to the camera may have, to ensure office privacy.
Can you send me the API info for how Transit Surfer accesses the NextBus data?
Another thing to check regarding reliability is to see how NextBus does in other cities where two or more transit lines occupy the same street… do the predictions get messed up when buses from different routes are closely ahead/behind one another? This will become important when the eastside loop opens and multiple streetcar routes begin sharing common trackwork.
And finally, can someone nudge NextBus to change the icons on their interactive map to make them resemble streetcars rather than buses? :-)
– Bob R.