Agile, Distributed, Site Development Structure

This idea came to me during interviews with an online media company – it’s amazing to me still that interviews are an amazing creative event for me – even if I’m not right/don’t get the job.

How does one build a process/environment for allowing constantly changing content and feature updates to a site without having to encompass all of that design/development talent in-house? How does one provide the inspiration and experimentation within a design/development team that allows an in-house group to continually pull ideas from outside “the house”?

The in-house group should always be concerned/focused on the infrastructure. The site is “physical plant” and the in-house group manages it, upgrades it and allows others to use/interact with it. Often infrastructure is merged in with the responsibilities of editorial and new development and in the course of trying to make the “next best thing,” infrastructure falls by the wayside.

Additionally, those who are good at managing and maintaining infrastructure are not always people who are good at developing new features. One group looks at how to refine, streamline and reinforce what is currently there. One group looks to push boundaries and go off in new directions.

With the web continuing to move in the direction of small teams (and individuals) developing new ideas or “mashing up” existing ones, there is a growing track record of innovation coming from small groups acting outside of traditional in-house requirements & politics. Often, the only ones to see how you could combine two different technologies or pull together multiple sources of data are those who are far enough away that they do not hold a fixed view of the hole (or any pieces) and are not constrained by any politics or policies that hinder “wild” thinking.

So the idea is that you let each do what they do best. You focus the efforts of the internal infrastructure team to not only build/refine the core system/technologies, but you have them build a robust staging environment.

Staging environments are sorely lacking in most big sites in general, even though they provide in-house groups greater control and freedom over making normal updates as well as trying new things.

With this idea, one would take the staging environment even further. One would create a live site (code & databases) and then a robust staging enviroment with various levels – akin to having a “working” area and a “testing/approval” area before you get to the “live” area.

Even with all development done in-house, the ability to have these levels is invaluable for allowing multiple stages of development. Without this, one sees needed fixes & small updates having to wait for larger projects to be finished/debugged in order to roll-out the code as they end up deeply entangled.

Extending the 3 stage set-up even further, the “working” area could actually be an easily duplicated affair, such that you could set up 1, 2, or even X number of “working” copies with all code and either copies of live databases or live database copies with dummy data.

Having multiple working environments would allow multiple teams/groups to work on separate ideas or projects. These could be multiple attempts to solve the same problem or build the same feature. But they could also allow each group to build their feature on the current code base on their own schedule. Should team A finish in 2 weeks while team B is another month out, team a’s copy could be moved up to the test environment, approved and made live. Team B, would then have their working environment updated (if necessary) to match the new live environment. This would/could provide some sticky situations, but would be something accomodated in a project schedule that would be warranted by the overall site being able to see the improvement of team a’s additional without waiting on team b. This way, teams or projects which get stuck in either development or political log jams would never impact the development of other teams. A working environment could be flushed or archived if a project is killed or put on hold and a new copy would be rolled out for another project/team.

If this infrastructure were in place – which would benefit so many in-house groups – it would provide the perfect platform for the next innovation encompassed in this idea. And that is a ticket/job manager that would allow ideas originating from in-house or external teams to be evaluated, built and implemented to speed innovation, new features and cut costs & timeframes.

Imagine an in-house working meeting where the company decides it would be nice to add feature A to the site. It’s seen as a really small change. Should be no problem. But wait, the in-house crew is working on v2 of the site and have no time to get to your small feature. Or feature A involves SOAP calls and JSP integration but your staff is all PHP based. Why not post a “ticket” to the job manager and let a trusted network of external “teams” look and respond. You might just find that a 2 person team in a city 3,000 miles away is up and can get that built overnight. They pop open a working environment, build it and have it waiting in the morning for the project manager to review.

The job tool could require all teams to enter bids, get approval which would prevent multiple teams doing the same work, prevent teams from having to haggle for payment afterwards, etc. The tool could also allow for teams to tackle a project in the environment without direct approval and hope that they would be first, best, cheapest, whatever and would get their development integrated into the site (and of course paid).

This last type of approach also lays the groundwork for the true value of this system. Up until now, this is somewhat standard practice. A new feature/tool either gets built in-house or an external contractor makes a bid and develops. They work in a staging environment and then integrate it into the live site when approved. But what about ideas no one in-house expected or imagined? What if the idea/enthusiasm which might now turn into a little alternative site/tool to your own instead gets suggested/built as a new feature of yours?

In addition to the in-house team posting jobs/projects to the job system for all (in-house and outside teams) to peruse and take on, it would allow those outside teams to suggest projects/features as well. So an outside team has an idea to bring content from A into a tool that uses info from B. Nothing like it has ever been seen before. They post to the system and either wait for approval or go ahead and build it in their work environment. Next thing you know, a radically new idea – a true “out of the box” idea – has been developed for YOUR site and in a time frame and using resources you didn’t necessarily have or want on staff. The value/price is set in advance or afterwards and you move on to evaluating and integrating the next new feature your network developed this week.

Immediate savings are seen to the company in terms of reduced in-house staffing as well as the cost of development. If companies have been attracted by outsourcing projects domestically or abroad, why not enhance that ability.

Also inherent in this approach is a shift away from seeing updates to sites as new versions, which the size and responsibility that always entails (what’s the political implications of v3 slipping for 4 months and nothing new changing on your site?). Build smaller, build more often, build faster.

With that – why bother with large scale 3rd world outsourcing firms when the very talented developer down the street (or over in Silicon Valley) can manage to build it. Large-scale outsourcing works when you need large projects. They just transfer the slow moving progress from in-house to someone else’s in-house.

Bringing in contractors can work… but you’re still asking someone to come in and be on your time and fulfill your needs for a while. Why not let them stay on their own time, in their own place, in their own comfort zone? Then step back and watch how fast they work.

With something like this, you’d finally see large-scale sites have the ability to innovate at the pace of current web technologies and approaches. You won’t be seen as trying to be “with it” when you roll out version 4 of your site that just happens to include tagging even though that was in vogue months ago. You could have the new feature/technology on your site the day AFTER it was invented – by the person who invented it.

And finally designers/developers would have the opportunity and structure for reaching the “virtual marketplace” we’ve been told so much about. The one where they say it doesn’t matter where you’re located, whether you’re one person or a team. You can – alone or with others – be influential developers working with multiple big sites.

Only then will it make sense for me to pick up from San Francisco and plop down in a big house in the country and still make a living.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *