Bringing Evolution Development Back to Life

MODX Evolution is the reason MODX exists and is very important to this project, but the core team is focused on Revolution, not the legacy codebase. We do want to improve things for the MODX community dependent on Evo, by finding a mutually beneficial way to enable those interested to continue Evo development.

For it to work, we’ve provided a set of guidelines and standards for the community contributions below. These are the same standards applied for any contributions, whether from the core team or from the Community.

Ownership of the MODX Evolution Repository

Currently, only a few people have the rights to push to the official MODX Evolution repository on GitHub. This serves as a gateway to the community of contributors to submit their contributions from their forks for review and acceptance before being pushed for permanent inclusion in the official, upstream repository.

With the current workload of the MODX Team, it also serves as a barrier. That must change!

Photo: Keys by Richard GarsidePhoto Credit

To revitalize Evolution-ary progress, we would like to invite those interested to identify and agree upon a single liaison—an Integration Manager to serve as the gatekeeper to the official Evolution repository. This person would be responsible for reviewing, managing and merging Pull Requests and marshalling related tracker issues, according to MODX contribution rules and standards, and would have push access to the official repository.

This is an incredibly important responsibility, and the person the community decides upon for this role needs to adhere to and enforce the contribution rules and standards.

While the quantity of contributions to Evo are enormous, many unfortunately don’t meet the proposed quality or project standards, as outlined below. Many have some great improvements that could be accepted into the project if they were not combined with a number of other unrelated changes that are not acceptable for one reason or another.

Contribution Rules and Standards

For the last 7+ years, we’ve figured out what works and what doesn’t in managing a large Open Source project. The following outlines the key take aways.

The first, and most important contribution rule at MODX is: one Pull Request per feature or bug. This is the one thing that will do the most to alleviate the Pull Request bottleneck that currently exists in MODX Evolution. Here are the reasons why:

  • it is much easier for the Integration Manager to review the code and decide if it should be accepted or rejected
  • it is easier to decide or alter where it can best be integrated for the current release cycle(s)
  • it is easier to un-apply the change post-haste, and without affecting other features or bug fixes, if a problem is found after the integration has been pushed

In addition, each Pull Request must have a corresponding ticket in the MODX Project Tracker for Evolution. See http://tracker.modx.com/projects/evo/issues for the current open list of issues.

Other MODX contribution standards which reinforce the MODX philosophy include:

  • The contribution should be as extensible as possible when implementing feature requests
  • The contribution should provide a reasonably seamless upgrade path for existing users of a feature being changed in any way
  • The contribution must not break backwards compatibility:
    • of core API methods
    • of included Extras or their associated API methods
  • The contribution should follow MODX Coding Standards as defined here.
  • All code contributors (this means the author of the commit, not just the integrator) must sign a CLA as described here.

Finally, new feature contributions in Evo should not significantly increase the challenges in providing a migration path from Evolution to Revolution. Though we do not yet provide a direct service or official tool for doing this, it is something the MODX Team is committed to accomplishing. We also think this will help keep the community better-aligned on key goals, preventing duplicated or other wasted effort where possible for both Evolution and Revolution contributors. Additional resources and standards for MODX community contributions can be found here.

Let’s Get Started, Today

Please take a few minutes to review these standards and processes. We consider these essential for anything with the “MODX” label attached—based on our extensive experience leading the project, integrating contributions, and releasing the product. We’re ready and excited to see Evo reinvigorated, and we look forward to handing over the keys to the Community ASAP.

About
Jason is one of the founding members of MODX, and now holds the title of Chief Architect. Because of his love for green chile and the mountains, he resides in Taos, New Mexico, with his wife, daughter, and son. From Taos, he enjoys hiking, camping, snowshoeing, exploring, and photographing the American West with his family. Jason is also an avid drummer and capable pool player. Author of xPDO and architect of MODX Revolution, he leads the development and innovation team for the MODX Content Management System.

http://www.jasoncoward.com/


20 Comments


  1. iusemodx
    Apr 19, 2012 at 08:30 PM
    When reading the "Bringing Evo back to life again" forum post the Japanese community seem to have made significant improvements to Evo.



    Does this post mean that the modifications that they have made are not (in the eyes of MODX Technologies, LLC) acceptable to policy standards?



    Cheers

    1. Jason Coward
      Apr 20, 2012 at 09:36 AM
      The Japanese communities changes would be amazing to have incorporated into Evolution, but there are so many, there is no way we have time to review, QA, and integrate those pieces ourselves. This is why we are offering the keys to someone who wants to step up and help break the obvious logjam of contributions to Evolution.

      The changes still need to be reviewed and incorporated (or not) individually. There is no way that many changes to a single release could be QA'd and prepared for the Evolution community without any disruption to their working sites if we tried to incorporate all of the changes they have been making at once. What we must do is establish better communication with this group and start working to break the logjam and build momentum in doing this right way.

    2. Stefanie Janine
      Apr 19, 2012 at 11:33 PM
      I'm one of the community in Bringing Evo back to life and currently I'm managing the github repository where all the changes take place.
      Of course, that are a lot of changes by now.
      In my work to get Evo running with more databases, I also started to implement unit tests and PHPDoc tags. These changes are going all over the source of Evo, because whenever I touch code, I add PHPDoc tags.
      The unit tests are a more straight, because I started with the change of the database class.

      At the moment I cannot bring all that changes into one ticket, that does mean, there will be a very big pull request in the future.

      The community in the mentioned thread decided, that we take the changes of the Japanese community, because it is going in the direction, where we want to go. I haven't run any test on this code, but I will do this, after I've finished the database changes.
      That does mean, that the changes of the Japanese community will be part of the community changes. If there are problems with these changes, they will be solved by the community.

      There is another reason to integrate the Japanese changes: They have a real fork, that is on the market.
      This should not happen again, just because there is no time to work with the commits.

      Cheers

      Stefanie

      1. Jason Coward
        Apr 20, 2012 at 10:16 AM
        Stefanie:

        Since starting the MODX project we have followed the well-proven practice of isolating changes into manageable, incremental pieces so that as they get integrated, if any one of the changes introduces a bug or doesn't function as intended, it can be removed without affecting any other changes made. I know there are cases where several changes have dependencies on each other, and yes, there are small exceptions to the rule. But this is not one of those small exceptions.

        I certainly appreciate the scope of the effort to incorporate the Japanese communities amazing contributions, as well as those you and a number of other participating contributors are making, but this is an on-going problem and a result of bad process that has stacked up over too long a period of time.

        We are not offering this process as a way to say any of these changes or ideas are not good, or a way to hinder development. We don't want the communities' contributions to be in vain and this strategy is intended to bring the efforts into alignment with best practices identified by other Open Source projects of our scale and gained from our own experience with the MODX project.

        Let us help you make this possible.

        Sincerely,

        Jason

        1. Stefanie Janine
          Apr 21, 2012 at 09:30 AM
          Hi Jason,

          I know, that it is much easier to track single changes, I'm developing software for more then twenty years in several positions.

          One improvement of this changes will be that there are unit test, something, that I've done back to Smalltalk times with SUnit. I am writing unit tests for each function that is new and for any function in a class that I touch. That is something that did not exist at all by now for Evolution.

          To get a better scope on the software development, it might be very useful, to work with Jenkins.

          What the Japanese community has done, is a fork of Evolution, because they weren't pleased with the development process, including, that they came to the opinion that their changes where not wanted at all.
          With the huge changes, that the community is working on, this may end up in fork by the Evolution community and the Japanese community. I don't want to fork Evolution, I want to bring it back to one Evolution without forks.

          If you really want to have isolated changes, than nothing of what already is done, would have been possible.

          I also know, that all changes needs a review and I offer to work on this for the changes, that where done by others. I've talked about this with yama, who is representating the Japanese community, and we decided to start this, after I've finished the changes of the database classes.

          Though, going back to isolated changes is as to throw all changes in the dustbin. Tell me, how you want to communicate, that all the work should have been for nothing?

          Best regards

          Stefanie

      2. Keith Penton
        Apr 20, 2012 at 09:18 AM
        I'd like to at least help improve the documentation, so I tried to get to the CLA request - but the link on the RTFM page doesn't work - modx.com/developer/cla takes me to the home page, and modx.com/developer is a 404. Hunting about a bit in the modx.com menus did lead me to http://develop.modx.com/contribute/cla/, however, which is more promising!

        Re changing one thing at a time, I agree it's definitely the way to go for bug fixes and independent developments. And that developments must be compatible with existing installations. But I'm uncertain about the implications for more wide-ranging changes (e.g. switching manager round to use jQuery throughout) -- or is it simply the case that this proposal rules out any fundamental development of Evo, not just attempts to back-port Revo features?

        KP

        1. Jay Gilmore
          Apr 20, 2012 at 01:36 PM
          @kp52 Thanks for volunteering to help out with the docs and sorry for the issues with the URLs. They have been corrected and are now being redirected to ensure legacy URLs on external sites can find their way.

          Jason will more adequately respond to your second part on the purpose of the standards outlined in the email.

          1. microcipcip
            Apr 21, 2012 at 11:08 AM
            http://rtfm.modx.com/display/ADDON/Home -> Is it possible to separate Evolution and Revolution Addons?

            1. Jay Gilmore
              Apr 21, 2012 at 12:30 PM
              microcipcip That's a good idea but we may need to look at tagging as a solution as Wayfinder and Breadcrumbs documentation works in both Evo and Revo. I am sure more add-ons will be ported over time too. What do you think?

              1. microcipcip
                Apr 21, 2012 at 02:14 PM
                I don't have any issue with the addons section, but I can guess how confusing it must be for new users. I think tags can be a solution.

          2. Jason Coward
            Apr 20, 2012 at 05:09 PM
            The implications for pervasive changes are the same; I would say even more important in these cases. The bigger the change, the more critical the isolation is until it is ready to be integrated with the appropriate branch. We are not trying to rule out any fundamental development of Evolution, or even attempts to back-port features from Revolution that are practical and meet the other guidelines. But we also want to keep Evolution a quality application that keeps meeting the needs of it's existing user base until lessons learned from the development of both Evolution and Revolution spawn the next major version of MODX. We think the best way forward is to mentor a chosen Integration Manager in this strategy to help them do what we would be doing ourselves if we had the resources to do so.

          3. nick
            Apr 24, 2012 at 04:09 AM
            It's great to hear the official stance from the MODx organization. I understand and appreciate the need for quality control to maintain the strong MODx brand equity. That is really important.

            But to me the official stance does sound a little like a road block.
            "We are happy for the community to make improvements, just not too many and don't make it better than Revo..."

            The phenomenal response to the "Bringing Evo back to life again" thread that Stefanie started shows how popular and relevant Evo still is to the existing user base - and how excited these passionate users are about improving this brilliant CMS.

            It would be a shame if the community support is there, that we can't bring Evo up to speed by making some significant improvements at the start (such as the Japanese community's caching system to overcome the 5,000 doc limit, conversion to jQuery etc).

            1. Ryan Thrash
              Apr 24, 2012 at 08:10 AM
              We really are happy and looking forward to handing over the keys to the community for Evo. In fact, I think yama is starting to redo his all-encompassing work as individual pieces, which is just awesome, as are the other things being proposed.

              We've always been advocates and adhered to the rule of not breaking existing users' sites. When we develop we ensure changes/fixes are made individually with a reference to a corresponding ticket. New features are developed in isolated branches which can merged upstream once vetted.

              Actually, these are just solid best practices to follow for any software project—not artificial roadblocks to discourage innovation. We also acknowledge the amount of work it will take for the changes being proposed for Evo, but it really is the right thing to do.

              Can you imagine what would happen if many sites all of the sudden broke because a corner case wasn't addressed prior to a release? Trust me, it's not fun at all—we've been there when it's happened before, and following the above rules saved us!

            2. Jason Coward
              Apr 24, 2012 at 11:25 AM
              "We are happy for the community to make improvements, just not too many and don't make it better than Revo..."

              That is a gross misinterpretation of what we are trying to communicate. We are simply trying to enable someone to help manage the massive work of trying to resolve the differences between where Evo is and where specific contributions want to take it. If a branch is prepared and PR submitted where everything is validated and without breaking backwards compatibility or disrupting existing Evo sites in any way, then perhaps it can be accepted as an Evolution 1.1 release. We would simply be more comfortable, and I think the result will be better, if the work was done based on the process we are suggesting. Then each change can be evaluated on it's own merit and dealt with individually. If the changes are such that they are all dependent on another, than the dependency needs to be integrated first and validated itself. But I do not see how what Stephanie is doing is even related to yama's work. Those should in no way be dependent on each other. If they are, this needs to be discussed it detail and a real plan of action needs to be proposed, reviewed, and agreed upon.

              1. Stefanie Janine
                Apr 24, 2012 at 12:36 PM
                The work of yama and me is independent. There is a branch on my fork on github, where all submits of the Japanese community are accepted. Currently yama is working on translating all the Japanese git commit messages. That work is not finished by now.

                yama and I will publish in the forums, when the branch is ready to test.

                I offer my work, to get unit tests on the cache classes. I'm writing unit tests for every class, where I get my hands on, that is, of course, my favourite work, but I think we need those unit tests. The coding standards mentioned in the text is for Revo, I guess, because by now, there are no unit tests and no PHPDocumentator tags in the Evo base classes.

                Thanks to git, I love branching and merging, therefore, we can create a branch, where we can mix my work and the work of the Japanese community. The available unit tests will help, to find problems. Fixes should be done on the original branches, to have the possibility to have the code separated.

                I've seen some functions, that where marked deprecated with plain text, I marked them with a PHPDoc tag. There should be a plan, how long deprecated functions should be available.

                1. Stefanie Janine
                  Apr 28, 2012 at 02:23 AM
                  I've got some questions about the process to get an Integration manager:

                  - This should be a clearly defined process
                  - This should include a legislative period
                  - A nomination by the community
                  - A voting process by the community with a definition who has a right to vote
                  (For example comitters can vote)

                  To get this clearer, one just have to have a look how other communities are doing this, Drupal is only one example.

                  In addition the process can be used for marketing, because MODX has something to say.

                  1. Stefanie Janine
                    May 05, 2012 at 05:39 AM
                    Even after one week there is no answer of the MODX team, how they would go on with their gatekeeper solution.

                    This article seems to be not of any interest for the MODX team. I've got a feeling of talking to bricks.

                    1. Ryan Thrash
                      May 05, 2012 at 09:58 AM
                      Stefanie,

                      Apologies for the lack of response. The entire team is quite maxed out preparing for CMS Expo in Chicago next week and unfortunately we dropped the ball here. We would like to discuss this with you and any other interested parties on how to move forward when we return after next week.

                      There is really no need for a large electoral process for this. Our hope is that an integrator would be nominated and proposed from the contributors involved in the revitalization project, and that work could start on integrating the innovations and change into Evo. If momentum and growth of participation increases and politics arise it may require a more formal process.

                      We look forward to helping foster the future of MODX Evolution.

                    2. Stefanie Janine
                      May 05, 2012 at 11:51 AM
                      Ryan,

                      I can wait some time, but I think, that a democratic structure is a must for a community process. It also shows the value of the community and leads to higher interest on submitting code. You can see this on community driven projects. A good example is MySQL and PostgreSQL. MySQL is company driven and PostgreSQL is community driven. PostgreSQL is the project with the most new features, while more and more people are thinking about MySQL as a software belonging to a dinosaur without ideas.

                      Good luck with the CMS Expo.

                      1. yama
                        May 06, 2012 at 04:37 AM
                        Hello, Ryan. > one Pull Request per feature or bug It is impossible though regrettable. I wanted you to keep pace with me early more. When early, it was able to do such. However, I have a feeling that it is not a serious problem. Whatever we may choose, it is because the present Evo does not necessarily get worse than now. It is such a reason, I leave it to you. I sent the coarse pull requests to Stefanie. I am glad when this is adopted. However, I am not troubled even if not adopted. I am going to redo an improvement over two years again. (At my pace ) Waiting forever causes exhaustion. However, we must be free. I would like to expect a good response from now on. I would like to tell this most to you. In order to continue mutual respect. Good luck and success with the CMS Expo.

                        This thread has been closed from taking new comments.