Joomlatools logo

Are white papers really helping Joomla?

18 February 2008

The Joomla! project is calling for white papers for the next big release, Joomla! 1.6. This is just one of the recent attempts to make the development process of Joomla! more structured, better organized, and more controllable. I’m all for that… kinda.

The Joomla! Bureaucracy

The white paper submission process, as well as the recently announced maintenance procedures, risk turning the project into one big bureaucracy. As ibnhafsun noted, that’s fine, as long as you can manage it. In companies, where employees are paid to fill in forms, this can work. Joomla! however is made by volunteers. Potential contributors will be scared off by reading the procedures, let alone actually having to go through all that. You also need people who read the submissions, filter them, organize them, etc. Does Joomla! have the manpower for these processes? And more importantly, where’s the fun in that?

Adding a button

Do we really need a white paper for adding a button? It’s just one of the proposals on the white paper forum. (Andrew, I don’t mean to downplay your proposal — it just happens to be a good example.)

What happens if ten great papers are submitted for additional buttons? That’ll clutter up the user interface big time. A better idea here would be to find someone with in-depth knowledge of usability. Someone who can tell us how we can make Joomla! more intuitive. Someone who can come up with a conceptual overview of what Joomla!’s UI should be like. I bet that person won’t tell us to ‘add more buttons’.

Incubators

With more of an ‘incubator’ approach, similar to what Zend and PEAR do, we can replace the white papers with a more organic process. If you want to build a certain feature, simply start coding in a separate branch of the codebase. Find some people who share your interest and make a first working version. As soon as you’ve got some basic functionality, you can let users try it, and get other developers to review your code. It’ll be much easier to discuss it, when people have had hands-on experience, instead of by reading your white paper. You’ll get feedback that’s actually worth something. When the code reaches a certain maturity, it can be integrated in the core (be it 1.6, 1.7, …). Or possibly, you’ll find out that it’s better kept as a separate extension and let the end-user choose whether he needs it.

The big advantage is that the different incubator projects can be developed more or less independently. New Joomla! releases don’t have to wait until all planned features are done. Instead, when an incubator project is stable and good to go, that can trigger a new Joomla! release. I’m sure this incubator approach will speed up Joomla! development, and result in higher quality code. I also believe this creates a healthy balance between developer-driven, and community-driven development. And more importantly, it’ll bring the fun back.

So here my proposal for the next stage of Joomla! development:

Less talk, more code!

  • AmyStephen

    Potential contributors will be scared off by reading the procedures, let alone actually having to go through all that.

    Certainly, I hear what you are saying, and yet, all evidence is to the contrary. We are starting to see doors open to the community in huge ways. There’s nothing like an invitation to participate that brings people in.

    The white paper approach is an invitation for considered opinion – analysis – diagrams – prototypes – connecting a broader set of ideas (and code!) to the development process. I am confident this *will* bring significant and meaningful improvements to Joomla!.

    People like you and me know Joomla! developers. But, the vast majority do not have that advantage and this process reaches well beyond “connections” by bridging to a potential contributor base in ways unimagined.

    Regarding your comment as to the advantages of a more organic and less formal approach, I honestly believe it is those types of efforts that will benefit most greatly. Consider the difference in perception between a white paper written by someone that essentially restates the obvious Joomla! needs ACL improvements with the likely response of a presentation provided by a developer with an install base of thousands of satisfied users. Of course, a well exercised extension, an organic solution that grew from need, will carry more weight.

    The issue today, Mathias, is that there has not yet been a path or an opportunity for those solutions to be taken into core. So, this *is* certainly forward movement.

    Of course, it’s impossible to answer your question as to whether or not white papers are helping Joomla! since we have only recently heard the idea proposed. I am encouraged; you are skeptical, but both of us are buckled in and are committed members of the community.

    These are good discussions, thanks for posting a link to your article on my site. It’s important for those who care about the project to talk about process and approach. Doing so is another good sign that we are emerging as a viable community.

    Thanks for all you do,
    Amy :)

  • pollen8

    I for one think the idea of the white papers are a step forward.

    Running fabrik, I can vouch that there are lots of people in our community with real problems, that Open source projects can or should be able to address, and that these people need a way of communicating those needs to the development team. Often those needs are not what us developers think they are in our little pixel ivory towers. At the start of the fabrik project I had similar feelings that this type of system gets in the way of “getting things done” i.e. coding. However, a change in mindset really helped me out. Instead of thinking that I simply needed to get x feature added, I’ve found equal benefit from saying “hey, Mr foo needs this feature because of this reason, which is kind of similar to what Mrs bar said last week, and if I did x then this would solve both their issues.

    The whitepaper format seems to be of the same level specificity as the forum’s bug reports board. Which is to say it stops people from simply posting “I want better acl”, (ala the old wishlist board which had a horrific level of noise on it, and was thus generally ignored by the dev team) Instead, the whitepaper submission requires the submitter to think, formulate and structure their idea in a communicable manner.

    I agree that if the dev team mechanically went though and implemented the top 10 features each time you would have a messy interface. However, I don’t realistically see this happening. Paraphrasing from the white paper submission process: “Proposals may be…altered slightly based on the way in which differing points of view from the community input can achieve a best fit result”. I’m positive that the selection process won’t take that long or be that cumbersome, and that the dev workgroup will have the sense to obtain the essence from the whitepapers.

  • Mathias Verraes

    I do agree that the papers are better than the ‘I want X’ feature requests. However, whereas requiring to write the specifications first leads to predictability, an agile methodology puts the emphasis on adaptibility. It acknowledges the fact that requirements change continually.

    Currently: Someone with sufficient insight submits a white paper. The user community discusses the proposal until something of a consensus is reached. The developer gets the green light, and starts working. A year later, a beta version of the feature is ready, but another 6 months pass by before a beta release of Joomla! can be done, because they’re waiting on some other project.

    With agile development: A lot of people discuss the need for a feature, without knowing exactly what it should do and how. A small team of devs start working, and build a basic _working_ version of a part of the feature — within four weeks! It can be released as a Joomla! 1.5 distribution with the one feature preinstalled. People test it, turn it inside out, give feedback, and another 4-week iteration begins.

    See http://en.wikipedia.org/wiki/Agile_software_development and especially http://agilemanifesto.org/ for a better explanation :-)

    I’ve had the pleasure of working on our ‘secret project’ using an agile methodology for the past few months, and it absolutely works. We’ve come up with features neither we or the people we work with would have come up with by merely talking about them beforehand.

  • masterchief

    Adding a button would go in the “Minor API Changes Thread”.

    I don’t think people realise this but I’m not so much interested in the “first post” but in the discussion that continues. I want to get into the head of how the community thinks. I want to know what their expectations are. I also want to help them draw out exactly what it is they want if they are having trouble communicating it.

    This is a new process. Will it be perfect? No.

    Will it be better than a dozen men and women sticking their finger up in the air to and guess what the community actually wants? I think the answer to that is an obvious yes.

    Will it have teething trouble? Most certainly.

    Will we improve on the process as it goes? Another resounding yes.

    Is it too early to judge whether the process is going to work? I’ll let you be the judge of that.

    We have actually tried the organic approach in a sense. Actually it’s more or less the way we’ve operated since I joined Mambo. Whatever the case, I think you must agree that something had to change with our current process. It wasn’t working.

    The reason we have gone for a more structure approach is simply because we have such a huge social responsibility. We have in excess of 2 million sites out there and if we break stuff in 1.6 to their detriment – well, you get the picture.

    Whether we like it or not we are now far beyond the hobby/boutique status that was and still is Mambo. Joomla! has taken a bad wrap for being caught up in it’s ivory tower and not being in touch with the community. This is a massive step away from that stigma.

    I appreciate your concerns and value your feedback. I understand that finally having “consultation” will be a shock to the system of some. It is my hope that learned chaps like yourself will get beside the community to help them through this new process. I could not be more thrilled with the responses we have received to date and am enjoying the dialog. And yes, it is fun.

  • AmyStephen

    Mathias -

    The biggest barrier to participation is simply not knowing how to do so. The “maintenance procedures” published are important communication enabling anyone in the world to learn how Joomla! intends to do things.

    I don’t see “bureaucracy.” I see transparency. People can learn how to monitor the work. In doing so, they also learn how to participate, too. That will yield contributors.

    No one is “merely talking.” Already, there is a growing work force of developers debugging, patching, testing, and committing changes. They are positive, engaged, and dedicated to improving Joomla!. This is transparent to the outside world. The work is recorded on the tracker. The email lists are open. Blog posts and process documentation published.

    These things are not resulting in a “slowdown.” Already, in only a matter of *WEEKS* Joomla! v 1.5.1 was released. Contrast that with where we have been, Mathias, and I believe it is obvious things are improving.

    Regarding “secret development”, I will simply say that historically, that approach doesn’t work so well with people. People want to be involved. They will invest if there is opportunity to participate. They will support what they invest their time in. They will defend what they have ownership in. This is basic human nature and our species has consistently behaved this way for as long as history has been recorded.

    I know it sounds odd, but, the truth is that by involving more people in the process, our progress accelerates. More importantly, greater involvement strengthens the long term support for the project, which I think is likely the most basic goal we all share for any decision made.

    Thanks, again,
    Amy :)

  • Mathias Verraes

    A little extra reading, to go with the links about agile development: http://en.wikipedia.org/wiki/Open_source_software#Philosophy

  • AmyStephen

    Perhaps we might might agree that Mozilla has been successful as an open source project?

    Here’s a link that might make clear the value of documenting processes and how doing so enables contributions, which accelerates progress!

    http://www.mozilla.org/hacking/

    We have so much to feel good about in this community. Great minds. Diverse talents. A wonderful community.

    Cheers! Amy :)

  • Spacemonkey

    Wasn’t JoomlaCode kinda the first step in this process? To me it was a place where folks could gather around a particular topic of interest to them, and collaborate as a team to implement a working version that suits their needs.

    If those needs are met well, then that code should be considered for inclusion in the core or future releases, depending of course on the general applicability/usefulness of each project. Perhaps that is the process the whole whitepaper approach should be trying to solve?

    For example, as both Mambo and Joomla we’ve taken separate projects and incorporated them into the core, and sometimes that included adding the original developers to the core team.

    However the process for evaluation of such projects to date was always “someone on the core team hops on their soapbox and tries to convince everyone else that this project should go into core”, which doesn’t exactly scale *blush*

    I’d love to see a public process for requests to include something into the distribution, or a public process to prioritize features on the roadmap.

    With those two things made available, then the whitepaper idea does feel like a very heavyweight process that doesn’t really result in code – but I’ve been wrong before ;-)

  • masterchief

    Mathias, I’ve had a read of Agile methodolgy. To be honest I would actually argue we are doing bits of it within the weaknesses clearly identified with this model. It mentions that one strategy is to separate the requirements gathering phase from actual development. I would say that’s what we are doing. Also, how do you overcome all the other weaknesses of the model that would be present in a project like Joomla!?

    To me Agile could be readily used by 3PD’s (something like DocMan for example), and it could be used for specific tasks with a development roadmap. In other words it would be fine for the micro-management of particular areas. However, I’m not seeing broad applicability to the process of rolling out versions and at the scale we do things as Joomla!.

    Whatever the case, debating which model that is to be used, whether purist or hybrid, is really missing the point and academic. This whole exercise is about putting consultation back into the hands of the community. As long as the chosen model is supporting that, it really doesn’t matter what it is.

  • Mathias Verraes
  • http://yangutu.com Dating

    Also, I’ve seen tons of examples that show how amazingly simple it is to place links on .edu sites, whether through buying them or taking advantage of open-source platforms like Joomla and WordPress in order to inject spammy links. …

  • http://yangutu.com Dating

    Also, I’ve seen tons of examples that show how amazingly simple it is to place
    links on .edu sites, whether through buying them or taking advantage of
    open-source platforms like Joomla and WordPress in order to inject spammy
    links

  • http://yangutu.com Dating

    Also, I’ve seen tons of examples that show how amazingly
    simple it is to place links on .edu sites, whether through buying them or taking
    advantage of open-source platforms like Joomla and WordPress in order to
    inject spammy links. …

  • http://www.spacify.com/modern_lounge_chairs-6-411.html Modern lounge chair

    Researchers will promote their “Fast Download but Eternal Seeding: The
    Reward and Punishment of Sharing Ratio Enforcement” research paper at
    conferences later this year. Torrentfreak has published the white paper highlighting this new file-sharing …

  • http://www.casamodern.com/indian-home-decorating.htm Decorating Small Space

    You get the feeling that the new governor is more interested in reading white papers
    and playing with balance sheets. He has, and while not a horse trader
    by nature, those around him are, and he seems comfortable with that, and
    it paid off. …

© 2006 - 2013 Joomlatools - Joomla! is a registered trademark of Open Source Matters, Inc.