What the Daily Universe forgot to explain

UPDATE (January 2012): According to the Daily Universe, Brandon Beebe has decided to discontinue Schedule Snatcher at BYU, although he’ll keep rolling it out to other universities.

BYU announced this week a few changes to the registration system that effectively crush an independent service called Schedule Snatcher. The article attacks this service rather pointedly, even if it doesn’t mention it by name.

What’s new
The new system makes two major changes:

  1. When a class is full, you enter a wait list and the system will automatically add you to the class when a spot opens up (first come, first served).
  2. Adding a class now requires you to fill in a reCAPTCHA.

This strikes down both of Schedule Snatcher’s salient features:

  1. Registering automatically for a class once a spot opens was the whole point of Schedule Snatcher, so this renders the service obsolete.
  2. Schedule Snatcher (or any other service) can’t even make requests on your behalf unless a human is there to handle the reCAPTCHA.

    UPDATE: Schedule Snatcher has already implemented a system to handle this. Apparently the registration system only prompts you for a reCAPTCHA once (consistent with my own experience yesterday).

Developer games
When you write to an application that doesn’t provide an API, you always run this risk. (And even when it does provide an API, you still run the risk. Twitter is a shining example of ruthlessly quashing developers. They’ve made developing Twitter clients a game of acquisition-or-obsolescence.)

This same thing happened to me with the BYU Bookstore’s “My Book List” application. I wrote a browser app that extracted the ISBN numbers from the page and looked up textbook prices on Amazon. It then displayed them so you could easily compare the cost to buy the book on campus or online. The Bookstore implemented this feature themselves directly into “My Book List”, rendering my app unnecessary.

I’m glad BYU is implementing these features, both the inline price comparisons and the built-in wait list. But it makes students more wary of developing tools on top of BYU’s systems that would make their lives easier.

Ambiguous claims

Jeff Bunker claims in the Daily Universe article that the new wait list feature “will make sure the long-existing practice of students holding and registering spots for other students will no longer be effective” (as if it were a rebel uprising). The reporter’s unfortunately ambiguous writing fails to explain how that would happen, making Bunker’s claim seem unrelated. Here’s how it works:

  1. You go to sign up for a class. It’s full, so you add yourself to the wait list.
  2. Someone else drops the class.
  3. The first person in “line” on the wait list automatically gets added to the class.
  4. Hopefully, you eventually get added to the class if enough people drop and move you up the wait list.

The reason this system can prevent upperclassmen from saving spots for their friends is that the person dropping the class has no control over who gets the spot he just vacated. It goes to the highest bidder.

Playing catch-up

All the same, it’s nice to see this “significant service,” as Bunker puts it, being provided in the platform itself, even if it’s merely a response to someone else’s innovation.

  • http://mchasej.tumblr.com CK

    I’m glad that they’ve made the changes.  Otherwise I was going to create an open source Java (or something else as easily portable) program to do the same thing, and advertise it for free.  I think it’s wrong to charge $70 for such a service.  (And probably not exactly in agreement with University policy, but that’s, at best, a grey area.)

    More than anything though, this is a good move on the part of the University to protect students.  It’s a very bad idea to give out your university username and password.  Anyone who used direct deposit effectively gave the guy their bank account number.  Not to mention their social security number, home phone number, address, parents’ names, and a ton of other identifying information. Oh, and best of all?  He’s tying it all to their Facebook account.

    If any one of the people who have used his service were to have their identity stolen, they’d be the first and biggest suspects.  I doubt they considered the enormous liability they took on when they opened up the site to the public.  In the end, the University might have kept these guys out of trouble in the long run.

    Besides, their idea wasn’t new.  They didn’t create the first script to grab classes when they opened up; it was just the first foolish enough to go public.  I don’t think BYU copied anything they’ve done, but they realized that if they didn’t meet a need of people to try and get into a class when there was an opening, people would try to do it themselves.  (Necessity is the mother of invention.)  Better that BYU do it right (and fairly) than let students put themselves in a dangerous situation.

    Anyway, that’s my take on the situation.  I think BYU did the right thing, and I think the changes only benefit (and protect) students.  I hope I didn’t hijack your post, but wanted to share by thoughts with ya.

  • http://globalconstant.scnay.com/ Steve Nay

    Thanks for your comments. I quite agree that BYU did the right thing. Also, I didn’t realize Schedule Snatcher was charging and collecting bank account information. The security and legal implications of running a business like that are staggering. Writing a script like that for your own use is a different matter. I’m just glad I don’t know anyone dumb enough to actually use this guy’s service.

    It’s good to see that BYU recognized the student demand for that functionality and implemented it themselves. This is a much more sensible and secure way to do it.

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    Great post.  I love reading peoples opinions about my website.  It was a great project but it isn’t dead yet, we are already working on a fix for the captch business.  

    @mchasej:disqus , It’s important to understand that you don’t have to be the first at something to become the most successful.  We also never collected bank account information or social security numbers.  There is actually very little important information like that on your route y account, probably less useful information when compare to Facebook.
    It’s interesting to see all these changes happening.  It really shows that progress is only made when there is competition.

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    “I’m just glad I don’t know anyone dumb enough to actually use this guy’s service.”
    A little rude . . . If BYU hadn’t made the changes it would have been dumb not to use the service.  With over 5k users, it would have been near impossible to get the classes you wanted without it.

  • http://globalconstant.scnay.com/ Steve Nay

    My statement was too blunt, you’re right. I merely mean to point out that the security implications of divulging a password to anyone are serious. Even if I knew you personally and trusted you, I would still be wary of giving you a password simply because your database could be compromised by an attacker. You’ve taken on a lot of responsibility.

    I didn’t know you had so many users; you’ve been quite successful. Competition is indeed good for progress.

  • Eric

    The only concern I have about the situation is that if BYU manages to completely shut down Schedule Snatcher, then innovation stops. What you see on ScheduleSnatcher.com today is only a small portion of what it will be tomorrow. We’ve come up with a large list of new features that have yet to be implemented that will completely revolutionize the way a college student registers for classes. But nobody but us knows about them. BYU may think that MyMAP equals Schedule Snatcher with the new changes, but they see only what ScheduleSnatcher.com offers right now. They’re only seeing the tip of the iceberg. There’s really much more to be done. So as long as we can get past the captcha, we’ll continue innovating and BYU will continue “playing catch-up”. If they somehow redesign the captcha so we can’t get into the system anymore, innovation ends and, well, I guess it’s their loss.

    Regardless, we’re happy to have been a catalyst to a somewhat improved registration system at BYU. We hope to continue making it better.

  • http://mchasej.tumblr.com CK

    @Brandon, Of course your scripts aren’t programmed to go out and steal SS number and bank account information.  But the information is there for the taking if you’ve got their username and password.  (If they’ve worked on campus, the information is easy to get.)  And like Steven points out, it might not be all y’all that take it, but someone might steal your database and then steal people’s information.  And like you point out, there’s a lot of info on Facebook too–and you’ve linked all that information with the information accessible through their Route-Y.

    I’m a bit curious: has S-Snatch been reduced to checking at midnight of a person’s registration and either registering them for classes or putting them in wait lists?  I mean, with the new wait list feature, it’s no longer useful to re-check classes for dropped seats so that you can register–it’s most beneficial to simply put yourself in the wait list as soon as you can and pray that enough people drop the class.
    Beyond all of this technical stuff, what if the University were to come out with a new Terms of Service that contained a clause forbidding the use of services like S-Snatch?  They can obviously track users of the service easily, and they could flag them and drop every one of their classes (if the hypothetical new terms said they would do so).  They could also block your IP if they decided that your service was against those terms.  (Does anyone know where such terms now exists?  I tried looking and couldn’t find them–but I didn’t look super hard.)Oh, and to the statement “if BYU hadn’t made the changes it would have been near impossible to get the classes you wanted without it.”:  I went through my entire undergraduate education and was in every single class I wanted.  Granted, I had zero desire to be in Bott’s mission prep class, but still: got into every classes I needed.  And the kicker?  I never went on at midnight, and a couple times, I forgot until the next day.

    And @Disqus, why must you default to a ordering that makes all of the comments make no sense? Silly Disqus.

    I hope that everyone understands that I’m not trying to cut down anyone, except maybe Disqus and their default comment ordering.  I’m just wary of a service that has access (potentially) to too much of my personal information.  I stopped using Mint.com for the same reasons.  And it’s an interesting topic.  And the conversation has gotten interesting.

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    Go check your RouteY information.  BYU is never going to display in plain text your ssn or your cc number.

    As far as linking your routeY and your Facebook, it could be an amazing feature.

  • http://globalconstant.scnay.com/ Steve Nay

    How _do_ you take payment, anyway?

    Steve Nay

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    Payment is actually all done through Paypal.  I don’t even have the Paypal API set up, it just goes directly to the Paypal website.

  • http://globalconstant.scnay.com/ Steve Nay

    @mchasej:disqus You bring up an interesting point. I trust Mint.com right now, perhaps because they’re an established player in the financial software business (Intuit), and perhaps because they openly display evidence of third-party security audits. If they were compromised, I’d be in big trouble. Yet I trust them. So why not a homegrown service like Schedule Snatcher?

  • http://globalconstant.scnay.com/ Steve Nay

    Gotcha. I figured it would be something like that.

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    “what if the University were to come out with a new Terms of Service that contained a clause forbidding the use of services like S-Snatch”
    As far as BYU goes, there isn’t anything Schedule Snatcher is doing that is illegal or against their policies.  They do have in there online policy something that says a student can’t give out his/her Route Y credentials though.

  • http://mchasej.tumblr.com CK

    They did change the Personal Information section of Route Y to star out most of the SS#.  However, if you’ve worked on campus, you can access your W-2, which, of course, displays your SS#.  And most people that work on campus use direct deposit (or those who have had scholarships/loans/grants).  Those people also have their bank account number in the direct deposit section, which can be seen in clear-text (in the browser/PDF).  Of course, all of this information is sent over SSL. And it’s all safely protected by your Route-Y ID and password.  Unless you give it out…  Oh wait…

  • http://mchasej.tumblr.com CK

    To answer your question, I’ll first refer to your own comment: “they openly display evidence of third-party security audits.”  And they have a Privacy Policy.  And a Terms of Service.  And security professionals.  And professional developers.

    S-Snatch needs all of those things, starting with the privacy policy and moving forward.  When you whois the domain, it’s behind BlueHost’s privacy registration.  It should lead me to an LLC or something.  (@Brendon, If you’re going to keep developing the service, you guys should really form an LLC and do everything in its name, move the domain registration to it’s name, etc.  An LLC would protect you from liability, to a degree, and that could be very important.)

    In short, Mint.com is a professional product that I can trace back to a professional entity who would (and could) assume liability if something disastrous were to happen.  Schedule Snatcher feels like a basement project.  There’s no way that it could handle the fallout if a security breach were to happen.

  • http://mchasej.tumblr.com CK

    Not to be nit-picky, but you kind of skirted the important word in my question:  ”new”.  What if their terms of service changed (terms are subject to change) and said, much more professionally, of course: “Thou shall not use an automated service that repeatedly checks class registration and registers ye for a class (e.g. schedulesnatcher.com).  If such a service is detected as using your NetID and password, registration privileges will be removed and all classes will be dropped.  The use of such services compromises the system stability, security, and purpose.”

    I mean, that’d be a pretty heavy handed blow, but what if?  I mean, they’d have complete grounds to block your service.  It probably does expose the system to a higher load, and if it were to become more popular, could cripple the system entirely.  If your service makes the entire system unfair and unusable, what then?

    But to my real question:  What if they block you, with terms of service or via IP?  How would something like that factor into your plans?

  • Guy

    “We’ve come up with a large list of new features that have yet to be implemented that will completely revolutionize the way a college student registers for classes.”
    Lucky Steve Jobs left something to be “completely revolutionized.” It seems like he was doing that a lot.

    Steve Jobs quotes aside, I’m wondering if any of your rest-of-the-iceberg revolutionary features just increase ease of use without decreasing fairness, or if Schedule Snatcher will always be a “money talks” application.

    Also, it seems like it wouldn’t be too hard to distribute a program to individuals who pay for your service where they can run the program and log in locally and get the same functionality without you having to collect and store their credentials (which goes against BYU’s online policy, as Brandon pointed out, of students not being allowed to share their credentials with anyone). Are those plans anywhere in the iceberg?

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    Well then there really isn’t much we can do.  If it is via IP . . . its fairly easy to route the requests though a proxy.   If it’s by terms of service then it will depend on the students themselves to decide if they follow it or not.  

    In reality, if they don’t want us, they can get rid of us.  Luckily there are thousands of other schools to go to. :D

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    Interesting, didn’t know you could access those things.

    I believe the weakest part of any system though is the people themselves.  It’s a lot easier to trick a person than to crack a password.  The more people you have with access to the system the weaker you become.  With Schedule Snatcher there is only one person with any access to any of the information. 

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    It would be a lot harder to distribute, debug, and develop an application to be installed on someones personal computer.   A website is really a lot easier to maintain.

    The idea of the new features on schedule snatcher will be to increase ease of use and fairness.  

  • John S

    Waitlisting has been on the enhancement list for a long time.  Many other universities already have this feature.  (UVU to name just one.)  Despite the damage it might do the SS egos, the timing had nothing to do with them in particular.  The change was in the works before SS appeared on the scene. 

    The real issue with SS that triggered the captcha change is that, like many automated processes, it puts too much load on BYU systems – kind of like a DOS attack.  This makes the system worse for students (actual humans) trying to register for classes because SS consumes too much resource.  So if SS circumvents the captcha change, BYU will be forced to find another way to block their automated traffic.

    Regarding surrendering one’s username and password, I would agree with Steve Nay’s original premise that you would have to be dumb to give up that information to just any third party.  Enough information is available that identity theft would be child’s play. 

    FWIW, I believe that BYU’s policy of not sharing credentials is to protect the individual, not the institution.  BYU never set out to “ruthlessly quash [outside] developers.”  BYU has a responsibility to maintain systems that are useable by our students, faculty and staff.  SS was diminishing overall customer satisfaction by robbing cycles from everyone but their customers.

  • http://globalconstant.scnay.com/ Steve Nay

    I wonder how things would be different if the registration system’s architecture were publish/subscribe rather than request/response. That would make it evented, so third-party applications like SS would simply subscribe to “spot opened in class” events rather than having to poll University servers to ask if a spot has opened. That would dramatically decrease the strain on the system while enabling students to write their own interesting applications.

    That would be coupled, of course, with OAuth-style authentication, eliminating the security concerns of divulging a password. Am I out in left field, or would the University consider writing the system this way?

  • Yoda

    No. It really is quite dumb to use this service. People don’t seem to understand how stupid it is to give out your net id and password.

  • Yoda

    So the innovation you’ve brought about is that now everyone has to enter captchas to register. You’re a real hero.

  • Yoda

    You are correct. There’s nothing groundbreaking here. SS is not even the first bot to charge people to use it. They just advertised it more (and got all of them shut down). This problem is that classes are scarce. SS doesn’t change this. SS provides nothing of value. For everyone who used SS to add a class there is someone else who didn’t get into the class. These guys are just trying to make money by wasting other peoples resources, and in the end making it worse for everyone. Might as well be running a spam email server.

  • http://globalconstant.scnay.com/ Steve Nay

    At least it provides some perceived value to the student with cash, something a spam email server doesn’t. But it is indeed a waste of resources for everyone.

  • John S

    I believe that BYU would be very open to creating a next generation system that uses a publish/subscribe model. I agree that would reduce the strain so systems like SS would raise far fewer eyebrows.  The questions are funding and timing.

    I agree with you about OAuth; I am surprised that more third parties don’t use the OAuth model.

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    What is funny about the whole thing is that Schedule Snatcher wasn’t a big waster of resources.  I purposely made Schedule Snatcher ping the server very slowly.  When the Administrator at BYU looked, Schedule Snatcher was pinging the server only ever 5-10 seconds, while at the same time there were 5-10 other bots pinging the server several times a second.

    Even the slowest of servers could handle Schedule Snatchers requests.  It’s the other bots that students had that had little to no regard for the Universities servers.

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    Isn’t that life for you?  For every winner there is a loser? Isn’t that a free market?  Find something people want and find a way to get them the solution.  

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    Yeah we actually talked to the Registrar at BYU about this very thing.  The problem like you said is resources.  Universities move slow and have very little money to invest in registration. 

    In their point of view it works, so why change it.  

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    In all honesty, the weakest point in any network is the people who administer it.  Also, the larger the project the harder it is to cover all your security risks.  Compare BYU to Schedule Snatcher, BYU has more administrators and a much larger system.

  • http://globalconstant.scnay.com/ Steve Nay

    Is there a way to use OAuth with CAS right now? CAS authentication of itself can only be used with .byu.edu sites, but if we had a more limited version that worked with external sites without exposing as much personal information, that would be interesting.

  • http://globalconstant.scnay.com/ Steve Nay

    John mentioned that the wait list feature had been in the works for a while, but it’s only now coming to fruition. I think it’s valuable to have independent developers like you pushing the envelope, simply because you are more agile than a bureaucracy-laden institution. You’re helping define the market and show what features students really care about.

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    I believe that’s a good point.  When it takes a university 2+ years to design a wait list. . . you know your in trouble.  

  • http://globalconstant.scnay.com/ Steve Nay

    Unfortunately, slow development seems to be the eternal curse of an organization as big as OIT. That’s another reason it would be nice to have a more open system that welcomes outside developers.

  • Yoda

    I’m sure you’re just trying to be nice, but I’m pretty sure that BYU knew people wanted to take classes (and wanted to take some classes more than others) before SS came along. If you take BYU at their word, waitlisting was in the works long before SS was on the scene. What sort of innovation is happening because of SS (other than everyone needing to enter captchas now)? Do you view spam filtering software to be an innovation brought about by email spammers?

  • Yoda

    I am curious if anyone here has any ideas on how to actually improve the registration problem. There’s probably a dissertation or two to be earned in figuring out how to apply economic theory to scheduling classes.

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    At one point we thought about making an auction system for classes.  Essentially spots to the class went to the highest bidder.  Everyone would get what they wanted.  The problem is, just like in a free market, the rich have the advantage.

    I’ve heard they are actually doing a similar system in the business school.  Each student gets a certain number of points which they can then use to bid on the classes they want.

  • Yoda

    What problem does this solve though? If BYU is interested in having a “fair” registration system (whatever that means) I don’t see how throwing out a signal to race for a spot is better than waitlisting or even just letting users stumble across openings. It would reduce automated traffic I guess, but there are other ways to do that.

  • Yoda

    I love this idea. It removes the race condition. It might be hard to figure out your schedule if you don’t know which classes you are going to “win” though.

  • http://www.facebook.com/people/Brendon-Beebe/692718596 Brendon Beebe

    Exactly, it kind of sucks.  I had this in my highschool.   We gave them a list of classes we wanted with alternates and the school created the schedule for you.  Makes everyone happy but gets rid of some freedom.  Guess it’s similiar to a big government vs a small government, fixed market or a open market.  Lots of economics behind it I think :D

  • http://globalconstant.scnay.com/ Steve Nay

    That’s not unlike the current system of staggering registration dates based on class standing. The currency is seniority, not money.

  • http://globalconstant.scnay.com/ Steve Nay

    My high school did the same thing. I was actually kind of surprised when I got to college and learned that I could choose my own schedule. It was a rather daunting task the first time or two.

  • Yoda

    I’d be willing to bet that the holdup has more to do with policy makers than developers. BYU, like its parent, is a very conservative organization. Changes usually won’t happen quickly no matter what they are.

  • http://globalconstant.scnay.com/ Steve Nay

    I’ve had that same question. When are they going to open up wait list registrations? Does that really make it any more fair? Probably; I don’t know.

    The benefit of having an evented model (this is getting into programming and systems design stuff, rather than economics) is that the system can be more efficient while still supporting internal and external developers. It’s just a different design decision, but it does open the door for people like Brendon to develop applications without wasting system resources.

  • http://globalconstant.scnay.com/ Steve Nay

    Exactly. Bigger organizations suffer from more policy makers. You bring up a good point about the conservativeness of the policy makers in this case. But despite BYU’s ties to the LDS Church, I don’t think it’s much different from any other university in the country in that respect.