DISQUS

louisgray.com: louisgray.com: Dealing With Capacity Overload and "Laying Off" Features

  • webomatica · 1 year ago
    Two quick thoughts - yes, I share your concern about data portability. I like to see a service provide an easy export of everything I've ever done with it, for many reasons - backups, moving to another service, and the worst case scenario that the service goes under. Google apps are good about this, and even Disqus has an export of all comments to an xml file.

    Second thought - then there are many services I use where I feel all my participation is valueless and I wouldn't care at all if the company went under and all my stuff was lost - Digg, Reddit, and even Twitter would fall into that camp. No interest in a file containing all my diggs up or down or the silly comments I'd leave on those sites. That's another way out of the data portability issue - only share data you can afford, or wouldn't care, about losing.

    So I suppose this is my roundabout way of saying - I am resigned to the reality that many of these new services will be unable to provide the reliable service we'd expect from a big company. So it seems I'm increasingly unwilling to hand over my critical data to any company that's going through growing pains, or isn't going to let us get that data out. I'll just share trivial stuff and always have a backup plan.
  • ChrisRossini · 1 year ago
    Here's how I'd answer some of Louis' questions:

    ***Do you, as a user, have an absolute right to all your archives, ever?***

    If, while signing up to Twitter, it is explained that I'll only have access to the last 15 days of tweets, and all others will be deleted, I see no problem there. However, once you have unlimited access, taking that away does not sit well. People don't like things being taken away. So the archive restriction needs to be spelled out on day one.

    ***Do you have the right to demand that once features are added, they never be removed?***

    I think if you're a paying user, you're probably entitled to a refund...since the original deal has been broken...(depends on the particular situation though)... If you're a non-paying user, you obviously have the right to move on to another service.

    ***And do services have an obligation to scale up their hardware and infrastructure to keep you happy? Would you help fund their upgrades?***

    If the service ceases to work, like Twitter recently, yes they do have an obligation to scale up...if they want to keep their users.

    If Twitter were to start charging its users, I'm sure there are plenty of people who would pay (depending on the price, and if their friends are going to pay too)...However, Twitter would lose a lot of people too...especially if there are other free alternatives like FriendFeed, Pownce, Jaiku...etc.
  • Dave Stanley · 1 year ago
    Just to clarify our intention, we're not going to dump the data entirely. We're going to archive it, and explore ways to let users have access to that data in the future. We haven't yet decided how far back to keep accessible data in the near term; whether it be 3 months, 6 months, etc. I'd actually be interested to hear how far back people realistically read their feeds. Our observations are that users aren't scrolling back too far, and maintaining this data for those few instances is hurting the experience for everyone else. So in the interest of understanding how Shyftr is being used right now, and maintaining an efficient system to accommodate that usage, we believe this move makes sense.
  • Louis Gray · 1 year ago
    Thanks for the clarification, Dave. The post started off by saying you were going to "track numbers" back a month, and that wasn't clear, as far as whether that meant an individuals statistics, or even if that meant users were going to be given a # (like their order of logins) and if someone didn't login, their data would be whacked.
  • Phil Glockner · 1 year ago
    SocialThing 'temporary disabled' services have been that way since SXSW in February, I believe. I check periodically in the hopes that they will turn something else.. *anything* back on, but so far, no joy.
  • Matt Galigan · 1 year ago
    Hey J. Phil and Louis...we had to disable them simply because of the sheer number of calls we were sending them.

    We call the services in a very different way than FriendFeed and others, mainly because our goal isn't just to aggregate you and your profiles, but rather, aggregate your friends on that service. The end result is a single dashboard with all of your friends' activity elsewhere.

    What this does, is cause a whole lot of extra calls that we have to make to the services. Some of the APIs that we call are prepared for this, and deliver most all of that information in a single, or maybe two calls. However, in the case of del.icio.us, Digg, Last.fm and YouTube, the process was like this:

    1 call for your activities
    1 call for your friends list
    x calls for x friends

    In a very short period of time, we might call a service like Digg 102 times in 1 second (let's say, if you had 100 friends).

    To remedy this, we've worked with the services to find alternative ways of calling them, as well as have completely re-written our architecture and ways that we call the services from our end. When we launch 2.0, it will reflect this, and all of those services will be added back in immediately. The new infrastructure also allows for us to call RSS based sites and services, as well as opens us up to many other services. We're expecting to launch the 2.0 features very soon (I can't give a specific date, but I can say that we're almost done testing, and pretty much just awaiting deployment now)

    I hope that could answer a few of your questions.
  • Louis Gray · 1 year ago
    Thanks for the update on SocialThing!. Your team got a lot of good coverage at launch, and one of the downsides to high expectations is a shorter fuse for users to accept things not being they way they had expected. An aggregation service like yours is clearly very data-intensive, and it's great to hear you're revamping the infrastructure. Looking forward to getting a note from you when this has taken place, so I can dive back in again.
  • Phil Glockner · 1 year ago
    Thanks for taking the time out to explain the idea going forward at SocialThing, indiekid! It definitely sheds some light on where you want to take the platform.

    I'm open for providing feedback on the existing site or the revamp, if you want it. You can email me at phil at scribkin dot com.
  • Robert Seidman · 1 year ago
    Cyndy got it right. The rules for developing good software are the rules for developing good software:

    http://www.profy.com/2008/05/31/web-20-killing-...

    Cyndy offers up that many Web 2.0 companies have seen themselves as exempt from best practices. Your post is merely more kindling for that fire.
  • CyndyA · 1 year ago
    :O

    Thanks Robert. I've gotten so much flack for that about how time-to-market is more crucial, and how development best practices take too long and cost too much money... if Twitter ends up failing due to the architectural inadequacies, that looks like over $15 million down the drain. Which is more expensive?
  • Robert Seidman · 1 year ago
    Cyndy, in truth I am a huge proponent of speed to market where product feature and functionality are concerned. I have a product development background and there is often a tendency to want to get the features and functions perfect before launching. I don't believe in that and think iterating is the way to go. But I'm talking about the front-end, customer facing stuff, that's fairly easy to iterate.

    It's harder to iterate on infrastructure, as Twitter is painfully learning. But, it has given the "mistakes were made" speech, now all it can do is attempt to fix it. I'm somewhat optimistic that it can be fixed, won't cost $15mm, and that money hasn't been flushed down the toilet. I'm not sure this winds up being awful for Twitter, sometimes being a victim of your own success is a good problem to have.

    That said, I still think your piece was spot on, but as ever it's easier to be a critic than it is to build good products! :-)
  • gregory · 1 year ago
    heck, gmail is getting weird, not big weird, but niggly stuff ... slow to load, normal letter goes to spam, the recent chinese electrical goods store "virus", sent, bounced, sent again, ok ... size may be a problem in the tech world, just like for trees or animals or countries
  • CyndyA · 1 year ago
    It's been in beta for four years. Obviously SOMETHING'S not right!
  • Phil Glockner · 1 year ago
    I always thought Gmail being in beta was so they could be completely free of responsibility if someone chose to use their service for a business-critical function.

    I mean, doesn't Google Apps use the exact same core Gmail application but in a non-beta way?
  • gregory · 1 year ago
    in nature scalability is never infinite ... a certain species grows only so large ... someday a phd thesis of three will be written on the correlations with natural systems and software systems
  • Andy DeSoto · 1 year ago
    That's right-- as others have said, those SocialThing! functions have been disabled for quite some time. You must not have taken a close look in a couple of months, eh! I'm really excited to see what the SocialThing! guys are cooking up for 2.0 because it's _usable_.

    Also-- my wireless is running a bit slow and it took over a minute to load the comments page. Yowza! Is that Disqus acting up?
  • Matt Galigan · 1 year ago
    Andy, thanks for the comment for sure!

    Louis, just thought I'd let you know that 2.0 is up and running, so why don't you come on back and check it out...much more stable, put some old services back in and tossed a few new features in there while we were at it. Check out the blog post describing everything here:

    http://blog.socialthing.com/2008/06/12/whats-ne...