Project Flow & Tracker basic (non-UI) installation profile on GitHub - Status update for early-December 2010

Project Flow & Tracker installation profile is now ready in basic, non-UI form at https://github.com/victorkane/ProjectFlowAndTracker/tree/6.x-1.0-alpha1 on the 6.x-1.0-alpha1 branch. Be sure to change to that branch after cloning the project or else before downloading the tarball. Instructions are on the README.md file for that branch.

The installation profile includes the following:

  • All required modules are included in the file system, including drupal (to be replaced by drush.make file)
  • The install profile enables all modules required to support Project Flow & Tracker content types, views and settings.
  • Organic groups settings are enabled and permissions rebuilt for it to take effect.
  • The pft_basic module is enabled, which uses the Install Profile API to:
    • Create all PFT content types, fields, and related settings (including og related)
    • Creates team member, project owner, and project leader roles and sets up Drupal permissions for them.
  • The pft_basic module additionally implements
    • All PFT views in code
    • Strongarm settings in code
    • Taxonomy vocabularies and terms in code (thanks to taxonomy_export)
    • Imagecache presets in code (supported by the image cache module itself)

This initial installation profile is suitable for those who wish to tinker under the hood and roll their own usable versions. You can create projects, biz stories, roles, user stories and tasks but you will have to implement your own navigation and project browsing on the UI level. Which might be just what you want.

In a week, certainly before the end of the year, there will be an additional installation profile option which will offer a full UI install including panels everywhere, the pages and settings as per the current staging site http://staging.projectflowandtracker.com/

Since it will be the holiday season, a complete "the making of" article will be published, sharing my Git based repository and sites development and deployment "everything in code" and features free and ctools exportables and install profile API based workflow (Again, features are great, but cannot be used for an agile dev process since they require too much arcane refactoring and are not a valid dependency aware packaging system).

This will constitute the first "off the shelf usable" version of the installation profile, and I will publish it on http://drupal.org and issue tracking can start to get fast and furious.

Then, a jQuery usability layer, with drag-and-drop, accordion regions and other features will be implemented in the next release.

Following which the current free SaaS site http://projectflowandtracker.com/ will be replaced with Chicago version (the 150 current user accounts will be preserved, and the old site preserved as long as anyone needs it at http://prealpha.projectflowandtracker.com/ ) way before Chicago.

At that point I will publish full documentation and videos on how to use Project Flow & Tracker, and around the installation profile project on http://drupal.org it is my hope a community will form in order to help implement the road map (including full bi-directional traceability via integration with Version Control API, Repoview and other best practices workflow solutions on d.o., as well as with popularly used repo sites; and upgrading to Drupal 7 (mid-2011).

"features are great, but

"features are great, but cannot be used for an agile dev process since they require too much arcane refactoring and are not a valid dependency aware packaging system"

Could you expand on this?

Please see two of my articles in which I reconsider features

Please see the most recent:

Towards a straight up Drupal development workflow for the working stiff – can't we all just follow industry standards?
http://awebfactory.com.ar/node/458

as well as the prior:
Features driven development -- the nitty gritty and thoughts on the future of sustainable "in code" development
http://awebfactory.com.ar/node/448

I'd be most interested in your comments,

thanks,

Victor

Was also confused by lack of features usage

I got the impression it was just a matter of developing a features methodology. (Having said that, you seem to have been looking into it pretty in-depth, so perhaps I'm mistaken.) I'm just getting into install profiles myself (I use profiler as opposed to install profile API, which to my understanding, is more concise and like the D7 install API), but I'll definitely need to check out your articles.

Anyhow, I find the largest benefit of features is revertability. So far, I find that building only the basics into the install profile is working best, since I think users appreciate the ability to monkey around on your install profile, and then be able to revert to the default when something goes awry or they want the defaults back. When keeping the config in the install profile inself, you'd need to reinstall to get back to how things were, I believe. This isn't particularly user-friendly.

But I'm definitely running into the issue of it taking way to long to rearrange features as you go. I understand that full pre-planning helps minimize this, but there are just so many ins and outs to features, sometimes you don't realize a sticky quirk until you stumble upon it, and then you have to reorganize your feature hierarchy on the fly...

Oh, and I noticed on a quick skim of your second article that you mention a drawback of "drush features update" where you can only work on existing feature items. I forget where I heard it suggested (a helpful slideshare presentation somewhere), but you can simply change items in the info file and then "drush fu" and it will rebuild with what you need. (Hint: as per this issue, be sure to start your feature with a ctools exportable based item included, or you'll run into a bug that's a real time-waster.) If you're working locally, you can version-control the myproject_features directory so that feature changes in "module/myproject_features/feature_myproject_feature1" are just a matter of a few git commands and you're A-ok. No tarballs or moving or even anything outside shell:

vim feature_myproject_feature1.info
blaberry blah blah
:wq
drush fu
git diff (if you're rightfully suspicious that drush fu might've done some unexpected)
git add .
git commit -m "easy-peasy"

Also, "drush features clone" allow you to forget about namespace from the top, and change organization without too much worry. Basically allows you to change feature machine names in 2 seconds if you need to.

Anyhow, I'll definitely be checking out those articles, and I'll try to give your install profile a spin and get you some feedback :)

Thanks so much for all the posting you do -- I've stumbled across your articles often in my wandering, and I've come to expect them to be totally informative!

This is just the kind of dev workflow debate I was hoping for...

When you say:

"But I'm definitely running into the issue of it taking way to long to rearrange features as you go. I understand that full pre-planning helps minimize this, but there are just so many ins and outs to features, sometimes you don't realize a sticky quirk until you stumble upon it, and then you have to reorganize your feature hierarchy on the fly..."

That's just the point. The link to the Funny Monkey people and their refactoring of features is meant to show that there is a solution to the features dilemma, but that this kind of solution is not compatible with an agile approach to development; whereas numbered updates to modules might be much easier to deal with.

Also the problem is that if a content type is a dependency for two features, it can only be included in one of them, and not in both as a kind of external reference. And that just plain goes against the way dependencies work in software development.

The reason features may work for a company is when everything they do is very similar, so that they can abstract out an underlying layer ("kit") which always works.

But that is too monolithic a solution.

Anyway, I am just exploring convenient ways of working, thank you so much for participating in this exciting discussion.

It's true that dependency

It's true that dependency management can cause problems with a Feature centric workflow but the main point I think is that just because you are using a quick point and click tool, you still at present need to be aware of the consequences of your actions and where it might be required to fill the gaps with custom update functions etc.

Let's take the problem that is most likely to come up if developing a site in an agile way. It's the same one you mention here, you've developed a node type and it is part of some feature but you have another feature that depends on it as well. Now it's true that the node type configuration can only like in one feature so the obvious refactoring is to extract that configuration out into its own feature that just manages the content type and which both features can now depend on. This sort of regular refactoring is not as arcane as you imply as long as you are aware of drupal's ins and outs.

My approach is to edit the feature's info file, removing all components (cck, strongarm, etc.) related to the node type, and running drush fu on that feature. This will leave the configuration in the database now and you can then reexport that config to a new feature.

This sort of refactoring tends to lead to a hierarchy of features with a main driver module/feature such as is mentioned in the nuvole presentation. Combine this with refactoring of update functions into install functions and a very minimal install profile that enables the driver/feature controller module and you can regularly refactor and blow away the database and reinstall to automatically clean up things like extra database tables from deleted content types etc. Of course if you are working agile then you will have production and staging instances to deal with before your design is done so you will have to write update functions to do some clean up for you but this is still very agile compatible and again the main point is that just because you are using features to speed up exporting of configuration you still should know about the nature of that configuration especially if you were developing agile.

Are there other examples that you are thinking of where features really sticks things up from an agile perspective?

Features may be at present the best way to export configuration

You make some excellent points, Boris. You may actually be right, given the current paucity of easy export interfaces and their lack of consistency. This is the debate I was hoping for, what you are saying certainly sheds much clarity on the subject, and anyone investigating their own Drupal dev process should certainly give what you are saying a good try. I know I have and will continue to. As long as people know what they are getting into really, and don't believe the flippancy with which feature driven development is supported sometimes as a silver bullet.

But things like the Ctools Bulk Export user interface are still in their infancy. Given that the dev version of Taxonomy Export, Image Cache, Strongarm, Panels and Pages, and Views all support export and have a UI to make that happen, admittedly the fly in the ointment is that there is no uniform way to do the exporting and a small amount of cut and paste is necessary in some instances. But this could change.

By far the biggest problem for my approach is that there is still no content types and fields in code (you can create and update using install profile api as I have done at https://github.com/victorkane/ProjectFlowAndTracker/tree/6.x-1.0-alpha1 where I also have a "thin" install profile, and delegate most stuff to install profile API driven modules (one at present, another one coming for the UI). But it is true you cannot have content types and fields in code as you can have views in code and even imagecache presets in code). Right now features is the only way to get content types in code.

What I am raising is the question of whether it might be easier to write drush commands to export stuff rather than be married to features.

But the "thin" install profile and feature hierarchies you are pointing to (and which Funny Monkey people use in voicebox install profile) could, I repeat, be the best bet at present, couple with the numbered updates espoused by the nuvole approach.

In answer to your question about additional sticklers from features for an agile approach, my biggest beef, apart from the strict refactoring overload (which would be acceptable, as you say) is that the underlying library "kit" is something which in agile should appear as a result of natural refactoring (as class hierarchies emerge over time in object oriented development) according to the application use cases, and certainly not as a monolithic and inflexible starting point. My "kitchen sink" may not be your "kitchen sink", according to the nature of the web app.

Again, excellent debate, my mentoring work reflects the fact that many shops are tooling up right now, so it's a good time to think.

Don't stop now

I'm really getting into this :-)