Features driven development — the nitty gritty and thoughts on the future of sustainable “in code” development

Features driven development in the Drupal sense involves placing code into versionable text files and packaging functionality into reusable components rather than relying on error prone user interface based configuration with scripting stored in the database along with the content rather than in code. It has been a great step forward for Drupal developers. It is simply a joy to once again be able to develop without infecting the code repository with database dump blobs. And to be able to install functionality in multiple servers running different applications irrespective of the content production workflow. But it is not without its contradictions and limitations, which I would like to mention in this article for the sake of discussion. However, while I will propose ideas for a better solution, I am mostly concerned with leveraging development in code right now by finding a comfortable process for using features driven development today, based on the promise of Adrian Rossouw's now classic "Database Is Silver, Code Is Gold" article The Development -> Staging -> Production Workflow Problem in Drupal.

Posted on: Last updated on: Written by:

Features driven development in the Drupal sense involves placing code into versionable text files and packaging functionality into reusable components rather than relying on error prone user interface based configuration with scripting stored in the database along with the content rather than in code. It has been a great step forward for Drupal developers. It is simply a joy to once again be able to develop without infecting the code repository with database dump blobs. And to be able to install functionality in multiple servers running different applications irrespective of the content production workflow. But it is not without its contradictions and limitations, which I would like to mention in this article for the sake of discussion. However, while I will propose ideas for a better solution, I am mostly concerned with leveraging development in code right now by finding a comfortable process for using features driven development today, based on the promise of Adrian Rossouw‘s now classic “Database Is Silver, Code Is Gold” article The Development -> Staging -> Production Workflow Problem in Drupal.

Features

The Features project page defines a feature: “A feature is a collection of Drupal entities which taken together satisfy a certain use-case.” Let’s see what this amounts to with a practical example, involving development of the latest release of Project Flow and Tracker project management software I am working on right now. Here is a snapshot of the use case, in this case expressed as a user story (that’s right, PFT is dev-booting PFT!):

A user story broken down into constituent tasks in Project Flow & TrackerA user story broken down into constituent tasks in Project Flow & Tracker

As you can see, in the Conversation section, there is an initial attempt (during analysis and design work) to break down the user story into constituent architectural “feature bits”; and this is formally reflected in final detail in issues list.

Because what’s the problem? Once you have a feature installed, the “drush feature update” command works great to update your “feature in code” in accordance with the changes you make on the site, so long as what you are modifying affects something already registered in the feature and you are not, say, adding another content type, view, or node view panels variant. But if you haven’t, and you are ready to create a new feature, you have to remember all the “feature bits” and not forget any, otherwise it’s Administer > Site building > Features and Recreate, then export to tarball, then unpack, etc. all over again. However, following some of the alternatives mentioned in this Open Atrium community blog post may help you avoid that in some cases (i.e. edit the feature module’s info file, then do a drush features update).

So while you can write down all the “feature bits” on a piece of paper in preparation for the grand moment, what I did was to formally break the user story down into a series of issues (tasks) in Project Flow & Tracker, in order to clearly document a list of items I would have to include when I was ready to create the feature, and in order to pave the way for future automation. Let’s see how that worked out in detail, in terms of creating the feature, updating the feature, and seeing how the feature relates to other features in terms of dependencies.

Creating the feature

Deciding to create the feature as things stood, the following “feature bits” had been documented:

  • Content type
    • Organic groups wiki post
    • Permissions for authenticated users to create and edit own
  • Taxonomy for biz story type
    • Created with machine name biz_story_type
    • Terms: problem, value, feature
  • Biz story view with block display
    • list_biz_stories
    • Note: this is part of the biz story feature, but the block will appear on a tab pane on the user story variant, thereby creating a dependency of that feature upon this one. It will specifically affect the node view user story variant.
  • Create biz story node view variant with same layout and blocks as others.

So I go to Administer > Site building > Features and click on Create feature.

  • Edit components
    • Content types: Biz story
    • Page manager: node_view: Biz story variant
    • Taxonomy: Biz Story Type
    • Permissions
      • create biz_story content
      • edit own biz_story content
    • Strongarm
      • [apart from the many already detected]
      • og_content_type_usage_biz_story
    • Views
      • list_biz_stories

Since many other “feature bits” were automatically detected, the end result was as follows:

 

Feature creation for Manage Biz Stories user story in PFTFeature creation for Manage Biz Stories user story in PFT

I clicked on Download feature, uploaded the tarball pft_manage_biz_stories-6.x-1.0-alpha1.tar to the staging server and unpacked it into the sites/all/modules/custom/features/pft_manage_biz_stories. There were the feature module files:

$ tree pft_manage_biz_stories/
pft_manage_biz_stories/
|-- pft_manage_biz_stories.features.inc
|-- pft_manage_biz_stories.features.taxonomy.inc
|-- pft_manage_biz_stories.features.user_permission.inc
|-- pft_manage_biz_stories.info
|-- pft_manage_biz_stories.module
|-- pft_manage_biz_stories.pages_default.inc
|-- pft_manage_biz_stories.strongarm.inc
`-- pft_manage_biz_stories.views_default.inc

Next step was to return to Administer > Site building > Features, where the new feature now appeared as a disabled feature, and select the checkbox next to the feature name, and click the Save settings button. The state changed to “Default” and all of my work on the user story was saved to code.

The usual bit of everyday Git, and my work was saved to github:

$ git status
# On branch master
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	sites/all/modules/custom/features/pft_manage_biz_stories/
nothing added to commit but untracked files present (use "git add" to track)
$ git add sites/all/modules/custom/features/pft_manage_biz_stories/
$ git commit -m "User story manage biz stories supported as feature module"
[master c45b161] User story manage biz stories supported as feature module
 8 files changed, 933 insertions(+), 0 deletions(-)
 create mode 100644 sites/all/modules/custom/features/pft_manage_biz_stories/pft_manage_biz_stories.features.inc
 create mode 100644 sites/all/modules/custom/features/pft_manage_biz_stories/pft_manage_biz_stories.features.taxonomy.inc
 create mode 100644 sites/all/modules/custom/features/pft_manage_biz_stories/pft_manage_biz_stories.features.user_permission.inc
 create mode 100644 sites/all/modules/custom/features/pft_manage_biz_stories/pft_manage_biz_stories.info
 create mode 100644 sites/all/modules/custom/features/pft_manage_biz_stories/pft_manage_biz_stories.module
 create mode 100644 sites/all/modules/custom/features/pft_manage_biz_stories/pft_manage_biz_stories.pages_default.inc
 create mode 100644 sites/all/modules/custom/features/pft_manage_biz_stories/pft_manage_biz_stories.strongarm.inc
 create mode 100644 sites/all/modules/custom/features/pft_manage_biz_stories/pft_manage_biz_stories.views_default.inc
pft@textworks:~/pft_staging$ git push origin master
Counting objects: 22, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (15/15), done.
Writing objects: 100% (16/16), 6.49 KiB, done.
Total 16 (delta 4), reused 0 (delta 0)
To git@github.com:victorkane/ProjectFlowAndTracker.git
   f3429f0..c45b161  master -> master

You can see the commit at http://github.com/victorkane/ProjectFlowAndTracker/commit/c45b16147ff472766681c5d51c2b34e6d421810a

At this point, part of the development had affected other features (for example, the project feature was overriden (on-site (in database) configuration was out of sync with the previously generated code) since a busines model view of biz stories was now appearing as a tab in the project node view variant. And other features were out of sync too. So I decided to update all of them via drush, using the drush features-update-all command (or “fua” for short):

$ drush fua
The following modules will be updated: pft_manage_biz_stories, pft_manage_prj_and_us, pft_manage_prj_issues, pft_manage_roles
Do you really want to continue? (y/n): y
Created module: pft_manage_biz_stories in sites/all/modules/custom/features/pft_manage_biz_stories                       [ok]
Module appears to already exist in sites/all/modules/custom/features/pft_manage_biz_stories
Do you really want to continue? (y/n): y
Created module: pft_manage_prj_and_us in sites/all/modules/custom/features/pft_manage_prj_and_us                         [ok]
Module appears to already exist in sites/all/modules/custom/features/pft_manage_prj_and_us
Do you really want to continue? (y/n): y
Created module: pft_manage_prj_issues in sites/all/modules/custom/features/pft_manage_prj_issues                         [ok]
Module appears to already exist in sites/all/modules/custom/features/pft_manage_prj_issues
Do you really want to continue? (y/n): y
Created module: pft_manage_roles in sites/all/modules/custom/features/pft_manage_roles                                   [ok]
Module appears to already exist in sites/all/modules/custom/features/pft_manage_roles
Do you really want to continue? (y/n): y

At this point I wanted to check that everything was synced up, so I issued a drush features-list command:

pft@textworks:~/pft_staging$ drush fl
 Name                                 Feature                 Status    State 
 Features Tests                       features_test           Disabled        
 PFT manage biz stories               pft_manage_biz_stories  Enabled         
 PFT manage project and user stories  pft_manage_prj_and_us   Enabled         
 PFT Manage project issues            pft_manage_prj_issues   Enabled         
 PFT Manage roles                     pft_manage_roles        Enabled

which showed that no feature was in state overriden (i.e. out of sync: drush had brought the code into sync with the changes effectuated in the site). Administer > Site building > Features showed the same story. So it was safe to commit to the repo again:

$ git status
# On branch master
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	modified:   sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.pages_default.inc
#	modified:   sites/all/modules/custom/features/pft_manage_prj_issues/pft_manage_prj_issues.views_default.inc
#	modified:   sites/all/modules/custom/features/pft_manage_roles/pft_manage_roles.views_default.inc
#
no changes added to commit (use "git add" and/or "git commit -a")
pft@textworks:~/pft_staging$ git commit -am "sync all features for invocation of manage biz stories"
[master 4b6cf5f] sync all features for invocation of manage biz stories
 3 files changed, 52 insertions(+), 19 deletions(-)
pft@textworks:~/pft_staging$ git push origin master
Counting objects: 24, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (12/12), done.
Writing objects: 100% (13/13), 1.60 KiB, done.
Total 13 (delta 9), reused 0 (delta 0)
To git@github.com:victorkane/ProjectFlowAndTracker.git
   c45b161..4b6cf5f  master -> master

Let’s see a simple example of this update feature thing, now, in slow motion.

Modifying and updating the feature

I now wanted to implement the task ‘Create “add user story” link to all biz story displays’ in order to implement a hierarchy relating biz user stories to certain child user stories. In that way, when I view a  biz story I will also be able to see a gallery view of associated child user stories (next task after we get through implementing this one).

The following feature bits are involved, which I documented in the issues for the user story I was implementing:

  • Add optional node reference field to user story content type pointing at parent biz story.
  • Position links (where the “add user story for this biz story” will appear) in node view variant for biz story.

Now, we know ahead of time that the implementation of this user story cannot be a feature or a part of a single feature, but instead affects two features: the biz story feature and the user story feature. The reason is that in software engineering, there cannot be a one-to-one relationship between implemented software components and user stories. Huh? Well, more on that later, let’s see how the feature development in code works in detail. It’s still really cool stuff.

First off, I add the node reference field field_us_biz_story to the user story content type specifying “Reference from URL” widget. I specify the autocomplete widget as fallback behavior when creating a user story in any other way, and “Add a User Story implementing this Biz Story” as the link title, and leave the return path as the previous page. Of course I select Biz Story as the content type that can be referenced. Upon correctly positioning the field and saving the user story content type, something very interesting happens immediately: when I list the features on the command line with drush, two features are already out of whack:

$ drush fl
 Name                                 Feature                 Status    State      
 Features Tests                       features_test           Disabled             
 PFT manage biz stories               pft_manage_biz_stories  Enabled              
 PFT manage project and user stories  pft_manage_prj_and_us   Enabled   Overridden 
 PFT Manage project issues            pft_manage_prj_issues   Enabled              
 PFT Manage roles                     pft_manage_roles        Enabled   Overridden

If only I could find out why the PFT Manage roles feature is now overriden. Fortunately there is, otherwise this would make for very dark doings indeed. We can invoke the drush features diff command (providing we have the diff module installed):

$ drush fd pft_manage_roles
Component: content
        'size' => 60,
        'type' => 'nodereference_url',
<       'weight' => '-3',
---
>       'weight' => '-2',
      ),
      'widget_active' => '1',
    ),
  )

Ah, because I moved the biz story node reference above the role node reference field (for user stories can be children of roles and of biz stories). I updated with drush features update, and then I did the same for the user story feature, and then updated it:

$ drush fu pft_manage_prj_and_us
Module appears to already exist in sites/all/modules/custom/features/pft_manage_prj_and_us
Do you really want to continue? (y/n): y
Created module: pft_manage_prj_and_us in sites/all/modules/custom/features/pft_manage_prj_and_us

At this point a git diff showed all the changes, which might be interesting for some so I shall include them here:

diff --git a/sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.features.content.inc b/sites/all/modules/
index a5533e8..ff91a18 100644
--- a/sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.features.content.inc
+++ b/sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.features.content.inc
@@ -191,6 +191,67 @@ function pft_manage_prj_and_us_content_default_fields() {
     ),
   );
 
+  // Exported field: field_us_biz_story
+  $fields['user_story-field_us_biz_story'] = array(
+    'field_name' => 'field_us_biz_story',
+    'type_name' => 'user_story',
+    'display_settings' => array(
+      'label' => array(
+        'format' => 'above',
+        'exclude' => 0,
+      ),
+      'teaser' => array(
+        'format' => 'default',
+        'exclude' => 0,
+      ),
+      'full' => array(+        'exclude' => 0,
+      ),
+      '4' => array(
+        'format' => 'default',
+        'exclude' => 0,
+      ),
+      'token' => array(
+        'format' => 'default',
+        'exclude' => 0,
+      ),
+    ),
+    'widget_active' => '1',
+    'type' => 'nodereference',
+    'required' => '0',
+    'multiple' => '0',
+    'module' => 'nodereference',
+    'active' => '1',
+    'referenceable_types' => array(
+      'biz_story' => 'biz_story',
+      'issue' => 0,
+      'page' => 0,
+      'project' => 0,
+      'role' => 0,
+      'story' => 0,
+      'user_story' => 0,
+    ),
+    'advanced_view' => '--',
+    'advanced_view_args' => '',
+    'widget' => array(
+      'node_link' => array(
+        'teaser' => 0,
+        'full' => 1,
+        'title' => 'Add a User Story implementing this Biz Story',
+        'hover_title' => 'Add a User Story implementing this Biz Story',
+        'destination' => 'source',
+      ),
+      'fallback' => 'select',
+      'edit_fallback' => 0,
+      'label' => 'Biz story',
+      'weight' => '-3',
+      'description' => 'Specify the biz story this user story's implementation will support.',
+      'type' => 'nodereference_url',
+      'module' => 'nodereference_url',
+    ),
+  );
+
   // Exported field: field_us_burned_points
   $fields['user_story-field_us_burned_points'] = array(
     'field_name' => 'field_us_burned_points',
@@ -558,6 +619,7 @@ function pft_manage_prj_and_us_content_default_fields() {
 
   // Translatables
   // Included for use with string extractors like potx.
+  t('Biz story');
   t('Burned points');
   t('Confirmation');
   t('Conversation');
diff --git a/sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.features.fieldgroup.inc b/sites/all/modul
index 2e11331..4a79595 100644
--- a/sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.features.fieldgroup.inc
+++ b/sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.features.fieldgroup.inc
@@ -79,7 +79,7 @@ function pft_manage_prj_and_us_fieldgroup_default_groups() {
         ),
       ),
     ),
-    'weight' => '0',
+    'weight' => '1',
     'fields' => array(
       '0' => 'field_us_confirmation',
     ),
@@ -118,7 +118,7 @@ function pft_manage_prj_and_us_fieldgroup_default_groups() {
         ),
       ),
     ),
-    'weight' => '-1',
+    'weight' => '0',
     'fields' => array(
       '0' => 'field_us_conversation',
       '1' => 'field_us_conversation_image',
@@ -158,7 +158,7 @@ function pft_manage_prj_and_us_fieldgroup_default_groups() {
         ),
       ),
     ),
-    'weight' => '2',
+    'weight' => '3',
     'fields' => array(
       '0' => 'field_us_priority',
       '1' => 'field_us_points',
diff --git a/sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.info b/sites/all/modules/custom/features/
index 9428742..7897d96 100644
--- a/sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.info
+++ b/sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.info
@@ -7,6 +7,7 @@ dependencies[] = "fieldgroup"
 dependencies[] = "filefield_sources"
 dependencies[] = "imagecache"
 dependencies[] = "imagefield"
+dependencies[] = "nodereference_url"
 dependencies[] = "number"
 dependencies[] = "og_views"
 dependencies[] = "optionwidgets"
@@ -20,6 +21,7 @@ description = "Basic management functionality for projects and their associated
 features[content][] = "project-field_project_end"
 features[content][] = "project-field_project_short_name"
 features[content][] = "project-field_project_start"
+features[content][] = "user_story-field_us_biz_story"
 features[content][] = "user_story-field_us_burned_points"
 features[content][] = "user_story-field_us_confirmation"
 features[content][] = "user_story-field_us_conversation"
diff --git a/sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.strongarm.inc b/sites/all/modules/custom/
index e387a06..14d9162 100644
--- a/sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.strongarm.inc
+++ b/sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.strongarm.inc
@@ -157,16 +157,16 @@ function pft_manage_prj_and_us_strongarm() {
   $strongarm->name = 'content_extra_weights_user_story';
   $strongarm->value = array(
+        'format' => 'default',
     'title' => '-5',
-    'body_field' => '-2',
-    'revision_information' => '5',
-    'author' => '4',
-    'options' => '6',
-    'comment_settings' => '7',
-    'menu' => '3',
+    'body_field' => '-1',
+    'revision_information' => '6',
+    'author' => '5',
+    'options' => '7',
+    'comment_settings' => '8',
+    'menu' => '4',
     'taxonomy' => '-4',
-    'path' => '9',
-    'og_nodeapi' => '1',
-    'list_issues_node_content_1' => '8',
+    'path' => '10',
+    'og_nodeapi' => '2',
+    'list_issues_node_content_1' => '9',
   );
 
   $export['content_extra_weights_user_story'] = $strongarm;
diff --git a/sites/all/modules/custom/features/pft_manage_roles/pft_manage_roles.features.content.inc b/sites/all/modules/custom/fea
index dbe8a5e..f4eddc6 100644
--- a/sites/all/modules/custom/features/pft_manage_roles/pft_manage_roles.features.content.inc
+++ b/sites/all/modules/custom/features/pft_manage_roles/pft_manage_roles.features.content.inc
@@ -67,7 +67,7 @@ function pft_manage_roles_content_default_fields() {
       'fallback' => 'select',
       'edit_fallback' => 1,
       'label' => 'Role',
-      'weight' => '-3',
+      'weight' => '-2',
       'description' => '',
       'type' => 'nodereference_url',
       'module' => 'nodereference_url',

So, long winded, but you can see how the content type is affected in the one and the cck field in the other, along with strongarm grabbing some variables. So we can feel more confident that we are actually saving our work in code, since we can see clearly how that is happening.

I viewed a biz story, but the link didn’t appear: I had goofed, forgetting to specify that the link should appear in both teaser and full node displays. So I fixed it and once again:

$ drush fl
 Name                                 Feature                 Status    State      
 Features Tests                       features_test           Disabled             
 PFT manage biz stories               pft_manage_biz_stories  Enabled              
 PFT manage project and user stories  pft_manage_prj_and_us   Enabled   Overridden 
 PFT Manage project issues            pft_manage_prj_issues   Enabled              
 PFT Manage roles                     pft_manage_roles        Enabled  
$ drush fu pft_manage_prj_and_us
Module appears to already exist in sites/all/modules/custom/features/pft_manage_prj_and_us
Do you really want to continue? (y/n): y
Created module: pft_manage_prj_and_us in sites/all/modules/custom/features/pft_manage_prj_and_us                         [ok]

So the link appeared ok now. But I edited the node view variant for the biz story so that the links appeared above the created and modified metadata fields to complete the task:

$ drush fl
 Name                                 Feature                 Status    State      
 Features Tests                       features_test           Disabled             
 PFT manage biz stories               pft_manage_biz_stories  Enabled   Overridden 
 PFT manage project and user stories  pft_manage_prj_and_us   Enabled              
 PFT Manage project issues            pft_manage_prj_issues   Enabled              
 PFT Manage roles                     pft_manage_roles        Enabled              

pft@textworks:~/pft_staging$ drush fd pft_manage_biz_stories

Component: page_manager_handlers
              'pid' => 'new-6',
              'panel' => 'main2',
<             'type' => 'node_created',
<             'subtype' => 'node_created',
---
>             'type' => 'node_links',
>             'subtype' => 'node_links',
              'shown' => TRUE,
              'access' => array(),
              'configuration' => array(
<               'format' => 'large',
<               'context' => 'argument_nid_1',
                'override_title' => 0,
                'override_title_text' => '',
>               'teaser' => 1,
>               'identifier' => '',
>               'link' => 1,
>               'context' => 'argument_nid_1',
              ),
              'cache' => array(),
              'pid' => 'new-7',
              'panel' => 'main2',
<             'type' => 'node_updated',
<             'subtype' => 'node_updated',
---
>             'type' => 'node_created',
>             'subtype' => 'node_created',
              'shown' => TRUE,
              'access' => array(),
              'pid' => 'new-8',
              'panel' => 'main2',
<             'type' => 'node_links',
<             'subtype' => 'node_links',
---
>             'type' => 'node_updated',
>             'subtype' => 'node_updated',
              'shown' => TRUE,
              'access' => array(),
              'configuration' => array(
>               'format' => 'large',
>               'context' => 'argument_nid_1',
                'override_title' => 0,
                'override_title_text' => '',
<               'teaser' => 1,
<               'identifier' => '',
<               'link' => 1,
<               'context' => 'argument_nid_1',
              ),
              'cache' => array(),
$ drush fu pft_manage_biz_stories
Module appears to already exist in sites/all/modules/custom/features/pft_manage_biz_stories
Do you really want to continue? (y/n): y
Created module: pft_manage_biz_stories in sites/all/modules/custom/features/pft_manage_biz_stories                       [ok]

All set to commit:

$ drush fl
 Name                                 Feature                 Status    State 
 Features Tests                       features_test           Disabled        
 PFT manage biz stories               pft_manage_biz_stories  Enabled         
 PFT manage project and user stories  pft_manage_prj_and_us   Enabled         
 PFT Manage project issues            pft_manage_prj_issues   Enabled         
 PFT Manage roles                     pft_manage_roles        Enabled         

pft@textworks:~/pft_staging$ git status
# On branch master
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	modified:   sites/all/modules/custom/features/pft_manage_biz_stories/pft_manage_biz_stories.pages_default.inc
#	modified:   sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.features.content.inc
#	modified:   sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.features.fieldgroup.inc
#	modified:   sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.info
#	modified:   sites/all/modules/custom/features/pft_manage_prj_and_us/pft_manage_prj_and_us.strongarm.inc
#	modified:   sites/all/modules/custom/features/pft_manage_roles/pft_manage_roles.features.content.inc
#
no changes added to commit (use "git add" and/or "git commit -a")
pft@textworks:~/pft_staging$ git commit -am "implemented task for user story manage biz model: create add user story link to all biz story displays"
[master 378f941] implemented task for user story manage biz model: create add user story link to all biz story displays
 6 files changed, 89 insertions(+), 25 deletions(-)
pft@textworks:~/pft_staging$ git push origin master
Counting objects: 30, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (15/15), done.
Writing objects: 100% (16/16), 1.96 KiB, done.
Total 16 (delta 12), reused 0 (delta 0)
To git@github.com:victorkane/ProjectFlowAndTracker.git
   4b6cf5f..378f941  master -> master

Notice how “drush fl” is now part of the commands you run as part of the version control commit sequence, just like git status: to make sure there’s no unfinished business or loose strings before you make that commit. So that’s the feature driven development cycle.

Sustainable in code devlopment: What’s this about there’s no one-to-one relationship between user stories and reusable software components?

Modern multitier software architecture allows us to separate concerns when we are building a web application. For example, when there is a request to visualize a content item in Drupal, the request is handed off to a logical handler (module), which should be using some kind of API to read the necessary data from the database. And once it has assembled that data it should invoke the theming engine and its current theme in order for the final output to be rendered all ready to be sent back to the browser that made the request in the first place. So this separation of concerns means that the code responsible for each _tier_ are separate from code in the other tiers. And the benefits are greater simplicity of each separate piece compared to the single monolithic block, and hence fewer bugs and greater ease and flexibility in when it comes to modifying code. So, code is housed in different tiers. Hold that thought.

Now, modern software development processes are incremental and iterative.

Incremental in that they strive to frequently deliver a measure of usable value capable of being manufactured in a relatively short period of time in order to get frequent feedback from the client, avoiding the failure-prone “waterfall” approach, in which the whole kit and kaboodle is analyzed and designed and implemented in one go on the basis of a one-time-only capture of requirements. That greatly enhances the risk of the delivered product not actually fulfilling the client’s requirements.

Iterative in that the usable value is continually improved in ongoing sprints as required.

One really cool version of this is the concept of “thin vertical slices” presented as part of the presentation On The Importance of Done in DrupalCon CPH by the team that implemented the Economist Drupal development. “These are (usually) specified as user stories [having] corresponding acceptance criteria” and involve an individual “feature branch” in the version control repo.  But they don’t use features to get their work into code, to move work to staging or to release work to live production, but rather the alternative of running update.php on incremental install API scripts.

The best description to date of the benefits and difficulties involved in using features in Drupal development can be found in the announcement and documentation of the FunnyMonkey’s VoiceBox, “a community discussion and  aggregation platform” sponsored by the Knight Foundation. In the VoiceBox Community Install Profile announcement,  the following section throws a tremendous amount of light on this discussion:

“The minimal install sets up the base functionality for VoiceBox. This functionality includes input formats, WYSIWYG settings, roles, and taxonomy terms. Then, on top of this base, the set of VoiceBox features can be enabled. Within the features we have created, we split out the content types, the views that display data related to those content types, and the UI elements that present and organize the data collected by those views. Using this structure, someone could (as one example) do a minimal install, and then enable the base components of the Bio and the Posts features. Then, they could create their own UI for this data using Context, Panels, the theming layer, or any combination thereof.”

From our own discussion, we can recognize here a sophisticated multitier approach to features development, with a set of “base” features upon which a presentation layer tier can be erected.

However, as far as I can see, this could only come about either through a “waterfall” approach which carefully planned out beforehand the erector set of features, or through a number refactoring sprints which would shift elements around until whole features could be assigned to tiers. The latter seems to be implicit in their insightful article http://funnymonkey.com/building-features-for-install-profiles , where the alternatives of “one huge feature” (to solve the interdependency problem) was rejected in favor of the refactoring approach:

“[editing] the exported features so that they contained just the components we wanted them to have – this process included removing the overlaps, and maintaining dependencies on both modules and other features. This allowed us to create some base features that contain the central keys to delivering the functionality – things like content types, imagecache settings, fields, etc. Then, we created some extras features that contain – for example – various views, flags, and other mechanisms used to organize information on the site. Finally, we created UI features that contain the contexts and the reactions that display specific blocks on specific pages.”

So how did the Economist guys get to skip this time consuming step? The answer is deceptively simple: a slow accumulation on all tiers, with each “thin vertical slice”, not in features, but in repository code, which accumulates and is automatically refactored with each sprint. Eventually, the successive updates are probably refactored into a “final” installation profile for each release, I imagine.

Listening to Young Hahn’s explanation on the now classic Lullabot Podcast 87: Panels vs Context, The Cage Match! sheds additional light (/me LOL – I say “rockstar is as rockstar does”: the empiricist rockstars knew enough that they needed to take a position to see how to handle their own development in the future… but not enough to know which position to actually take… but enough to know that if they put Young Hahn in the same ring as Earl Miles some truth would out! Cool actually, you got to hand it to them). It’s easy to understand why Development Seed went the Features route: they already had their base (see kit and super kit and super super kit or build kit), as an ongoing abstraction of the kind of work they have been doing over the past few years, and the kinds of sites they were producing obeyed similar use cases. And it was more for producing sites on the website developer level, rather than allowing the end user to “roll their own” through re-use. In any case what I am calling “super super kit” is an article published just a few days ago by Young Hahn, http://developmentseed.org/blog/2010/sep/30/features-and-exportables-drupal-7, which confirms that some kind of kit, or base tier, is assumed for features driven development. One interesting reference worth reviewing made in the article is Nuvole’s Andrea Pescetti explains how to integrate Features into your workflow. It is also based on build kit. Not true small core.

So what am I going to do with Project Flow & Tracker development?I think I know where I would like to get to: being able, with each “thin vertical slice”, to split out my development work as exportables (between automatic updating of a usable drush make file plus install profile bits and/or component modules) which will accrue on each tier in a logical manner, instead of being lumped together in baseless monolithic features; and to split out the content workflow into content that can be deployed from content production site to content serving site, etc., in an entirely independent fashion (think newspaper that develops its content on one (say intranet) site and serves it on another).

Right now on the dev side I am thinking CTools Bulk Export of something like “thin vertical slices” into successive modules which will later be refactored for each release; although for the coming (very usable) alpha release, the apparently logical separation into five or six features based on actual develoopment will really constitute a single monolithic feature. I may find that it is not so difficult to refactor (edit) features into install profile code. Certainly you can follow on github to see what happens, and I will certainly continue to comment here. Hopefully there will be an ongoing substantive debate on these questions.

But the joy of commiting code as I work, and not blobs of database dumps, is here and now.

Unfortunately I cannot say the same for content, as mentioned above: I am actually using Project Flow & Tracker for its development, so the only way I can actually save that is… via database dumps. I am investigating the deployment module now for an sustainable workflow solution for content.