Permalink 1 note
I’ve never been a fan of being in the spotlight. Unfortunately, my contributions to Drupal eventually came with more responsibility and visibility in the community, which led to a great deal of being in the spotlight on one level or another. When asked to lead the HTML5 Initiative, my initial gut reaction was to say no, and I did. I pondered for weeks before finally agreeing to do it. Ultimately, I said yes because people in the community that I have a lot of respect for wanted me to do it, and I didn’t want to let them down. I felt that I could effect positive change and attract more front-end developers to participate in the process. I had just spent the last 1.5 years making enough changes to Drupal 7 to be among the top 50 contributors to Drupal core, and while it was a struggle on more occasions than others, I felt good about the work and I had no reason to believe that Drupal 8 would be different.
It was okay for a while, but during the last year, I’ve have some pretty crappy experiences with some people in the community and working with Drupal itself. I’m not going to get into much detail and I won’t name names in private either, so please don’t ask. I will say this… I started noticing things changing for the worse almost a year ago. Discussions often get heated because there are lots of passionate people involved with strong opinions. I’ve always seen this as a downside of participating and have had a low tolerance level for it from day one. And maybe it’s just that my perception of things has changed, but it seems to have gotten considerably worse over a short period of time. I’ve also experienced some relationships that were important to me deteriorate, which has been nothing short of depressing. I’ve been vocal about some of it, but for the most part it’s been a long, lonely road that has left me completely uninterested in contributing any more of my time to Drupal core.
Overall, my experiences with Drupal, both personally and professionally, have been on the decline for the last year. I’ve consistently become more frustrated, and had less and less to say, on Twitter and elsewhere, for fear of offending one person or another, or having my words be misinterpreted. I’ve been “called out” for speaking negatively on more than one occasion, and told that as a “leader” in the community I should not say such things. I’m now at a point where I rarely even monitor Twitter, the issue queue, or GDO. I cannot even tell you how many comments and tweets that I’ve composed only to never actually post them. It’s more than trying to not say anything if I don’t have anything nice to say, and I don’t like feeling this way.
Ultimately what it comes down to for me, and of course your mileage will vary, is that it’s no longer working for me. Contributing used to be fun, and I used to feel like my contributions effected positive change, but that’s no longer the case, and I can’t seem to get out of the rut to turn things around, so I’ve decided not to contribute to Drupal in any sort of official role anymore.
The reason this post exists is because I feel that I owe an explanation and apology for my absence to the members of community that contributed to and were counting on me to lead the HTML5 Initiative to the completion of Drupal 8. I’m sorry that I’ve let you down, but I can no longer do it. Not only has it been a trying and stressful experience on a personal level, it’s also been a drain on my finances that I need to rectify and that takes priority over everything else right now.
It’s a mess. Yeah, I said it. It’s time to admit there is a problem and get serious about fixing it. In this post, I’m going to explain what I think the problems are, and how I’d like to see them fixed.
During the last year, I’ve spent a lot of time working on the HTML5 Initiative. There has been an incredible amount of work done, by a truly awesome, and growing team. We’ve gotten a whole lot done so far. The HTML5 Shiv has been added to core, the major templates are all done, and work on Form API elements is moving along. We’re also doing CSS re-factors, at minimum separating administrative styles from the rest.
Still, I’ve found myself very unhappy with the theme layer. I see a lot of work going into WSCCI and now the Layout Initiative that is really exciting. What I don’t see is if and how that will actually affect the theme system we work with today. This has been really discouraging for me, mostly because I’d like to see a lot more than just markup changes.
I’ve also become skeptical as to whether or not improvements for PHP developers automatically translate into front end improvements. I feel that we got burned with the implementation of Render API. I don’t think this was intentional or anything, just sort of the way things are. I believe the problems that it attempts to solve are hard and the way it goes about it is good, but it was implemented so inconsistently that it ended up making things harder. Many are unwilling to even give it a chance. Without consistent patterns to rely on, many of us are lost. Had the entirety of Drupal’s code base been converted to use the Render API, maybe it would have been different.
We are Doing it Wrong
When I complain about the theme system, I’m often surprised by some of the reactions I get. Many developers know there is a problem, but some seem completely unaware of the problems we face, and the things (which they would consider hacks) we have to do to get the job done. So, in no particular order here is what I think is wrong:
Too Many Templates and Theme Functions
Creating a new template was the answer for everything in both Drupal 6 and 7. It has been done carelessly, over and over again, and encouraged as The Drupal Way. As a result, we have hundreds of them. This is just plain ridiculous, considering there are only 109 HTML tags, most of them are used in groups and many of them are text-level elements that wouldn’t exist in Drupal’s theme system. We’d never have theme functions or templates for
<span>, etc. At least I hope not.
Lack of Consistency
There is hardly any consistency in what we’re doing inside templates and theme functions. Templates are a mess of arrays, strings, render arrays, and random theme settings variables. Look no further than page.tpl.php, which is the first impression newbies get with Drupal. In order to change a given variable in page.tpl.php, you need to first track down how it was generated and then figure out the appropriate function that will allow you to change it, which could be one of SO MANY different things:
- THEME_process_page() if that doesn’t work or you’re looking for something that happens late.
- Theme settings tweaks in the .info file, and the UI, along with one of the above.
- THEME_menu_local_tasks_alter() if you want to modify contents of tabs or action links.
- THEME_menu_local_tasks() if you want to modify the markup of the tabs.
- THEME_menu_local_task() if you want to modify the markup for the list item and links for the tabs.
- THEME_menu_local_action() if you want to modify the markup for the list items and links for the action links. Oh, and don’t forget to wrap that one in a
because just printing the rendering the variable alone leaves you with list items but without a
- THEME_links(), THEME_links__main_menu() or THEME_links__secondary_menu() to change the markup for the navigation links.
- THEME_page_alter() if you need to hide or relocate regions or get at anything inside them.
Data structure is a nightmare, and clearly an afterthought. The variables in templates are complicated, unreliable, and chronically inconsistent and it all starts with the data structure. Unless you are a PHP Ninja and can deal with pre-render functions, good luck trying to change a table into a list or vice versa.
Drupal may be notorious for its large, complex arrays, but IMO they are not the problem, nor is the existence of data structure. I believe the fact that data structure is whipped up on a case by case basis and pushed straight to templates in many cases is the root cause for the mess of a theme system that we have.
Imagine how much easier Drupal would be to learn if the data structure was completely consistent? Imagine if you could just guess based on experience where the value you are looking for is? Imagine if you could transform the format provided for the output from plain text to a definition list by setting one property before rendering? Imagine if contrib developers didn’t have to write theme functions for everything?
Too Many Levels of Processing
We’ve got preprocess, process, pre-render and and there can be endless implementations of these. There are no hard and fast rules for when to use one over the other. Sometimes you learn the hard way that you need to use process instead of preprocess because some module decided to add a variable in the process phase, and nothing you are doing to override that variable has worked in preprocess.
Other times you’ll get burned by pre render functions. For example, a while back I was struggling trying to prevent the filter help text from printing on a comment form. Attempting to hide it in a preprocess, process function or a form alter implementation simply did not work because there is a pre-render and process function manipulating it. The only place it would work was in the template itself. Ultimately I had to implement hook_theme() to use a template for the form solely to accomplish a simple hide(), because the template was the only place it would actually work. It took me hours to figure this out because the data I had access to in the theme layer was what it looked like before before the pre-render function changed it. That’s way too much troubleshooting and steps to accomplish something that small. This sort of thing is just maddening.
Additionally, there is no shared processing phase across theme hooks. The default preprocess variables ONLY apply to template file implementations, and cannot be used for theme functions. This means if you need any of the default variables, such as attributes or anything else created here, you must recreate it in your own theme function implementation.
Doesn’t seem worth it at the end of the day to me.
Your Output is NOT Special
Sorry, but it’s just not. We need to stop treating output as a product of the immediate need of whatever module or API is generating it. The content should drive the output, not the system or API behind it. I’m not saying that we should remove identifiers from output, i.e. a class that labels a container a node or a block. What I’m saying is…
- Instead of creating new theme hooks for everything under the sun, we should have a common library of components and formats to pull from and utilize.
- Every container, for example, should be handled the same exact way, whether it’s for a region, node, form element, or whatever.
- We should stop abusing theme settings to manage content and leave that up to things like blocks, so that content can actually be managed by Drupal. It is a CMS after all.
We don’t have this today. What we do have is often abused and or not flexible enough, so modules create one-off implementations because it’s easier for them to get the job done. This ends up hurting everyone. It’s more code, more complexity, and the attempts to improve things little by little without any planning are making it worse.
What I Want to See
I’ve been thinking about this for a long, long time, and I’ve started to try and get a handle on what we could possibly do numerous times in the past, starting as far back as two years ago. Ultimately, I’d always be asked “How can we fix it” and always just gave up because the changes that would be required are just massive, and I don’t know if I have the technical aptitude to pull it off. Recently I realized that it’s never going to happen unless I’m able to communicate what I’d like to see somehow, so six weeks ago, I bit the bullet and started gathering my thoughts. I came up with the following.
Note: It’s still very rough, and not completely thought out. It also doesn’t aim to solve all of these problems, but I think it’s an improvement. Also, I am a theme developer, not a module developer, so please don’t destroy me if you see something you don’t think will work. All of this is up on GitHub and you are welcome to tweak it, log potential problems and send pull requests. Ok? Great.
Provide Two Structural Templates
- The main purpose of a container is simply to wrap Items (see below). They can be used for anything from regions to nodes to form elements. They usually contain a heading, whether visible or not, and can print ALL children (items) in the base template for that container (sort of like the way you can print and entire view inside views-view.tpl.php).
- Items are structured content, which are eventually styled via formats (see below) and or components. They do not print ANY wrapper markup by default, and can optionally be wrapped with containers.
Both of these templates contain the same variables, and the same data structure for as much as humanly possible. They also include $append and $prepend variables that allow content to be injected at the beginning and the end of templates. These are special variables to deal with things like Contextual Links and Shortcut links, which also contain structured content and should not be abused like #prefix and #suffix are today.
Use Formats and Components to Generate Output
- Formats consist of HTML markup as defined by the spec. Some examples of formats include, unordered or ordered lists, definition lists, tables, etc. Formats need to be extremely flexible. They are variants of the default item implementation, and the key to making them awesome is providing the same data structure across different formats so that they can easily be swapped to use one or another without having to manipulate data.
Some of the Potential Benefits
If we are able to pull this off the way I’m envisioning, we can drastically reduce the number of templates. We can continue to give the theme developer control over the markup without so many assumptions and a sane, reliable data structure and workflow, which IMO will make Drupal a lot easier to learn and a lot less frustrating to use. It should also allow us to use other theme engines, instead of being locked into PHP Template.
So, check it out. It’s not, by any means complete, but there are some examples that should give you a decent idea of what it would look like:
Then, join the relevant conversations on Drupal.org:
Here are the slides for the presentation I gave yesterday at Drupalcamp NYC 10.
Allow me to share the best cocktail I’ve ever had with you… :)
- 1 shot - Absolut Vanilla vodka
- 1 shot - sour mix
- ~2 - muddled strawberries
- ~3 - fresh mint leaves
Combine all the ingredients above, except for the Sprite into an 8 oz. glass. Fill the glass entirely with ice, and then fill the rest with Sprite. Mix it all up (in a Martini shaker if you have one) and enjoy.
Source: I learned of this drink at “B Bar” in The Borgata Hotel and Casino a few weeks ago.
This guide will show you what patches are and how to work with them in the context of the Drupal project (though it will likely be useful for any project). When you’re finished reading, you’ll be able to create, apply and revert patches like a pro. Here’s what we’ll cover:
- The Anatomy of a Patch
- Check out a project from the Git Repository
- How to Create a Patch
- How to Apply a Patch
- How to Revert a Patch
The Anatomy of a Patch
A patch is a document that shows the differences between 2 versions of one or more files. We use them for Drupal development along with version control (Git) to communicate changes in a way that is very easy to understand, share and review. This is an example of a dead simple patch that changes a single line of code. If it looks a little complicated, not to worry. We’ll break it down bit by bit.
diff --git a/modules/taxonomy/taxonomy-term.tpl.php b/modules/taxonomy/taxonomy-term.tpl.php index b515a9b..872d4cb 100644 --- a/modules/taxonomy/taxonomy-term.tpl.php +++ b/modules/taxonomy/taxonomy-term.tpl.php @@ -37,7 +37,7 @@ * @see template_process() */ ?> -<div id="taxonomy-term-<?php print $term->tid; ?>" class="<?php print $classes; ?> clearfix"> +<div id="taxonomy-term-<?php print $term->tid; ?>" class="<?php print $classes; ?>"> <?php if (!$page): ?> <h2><a href="<?php print $term_url; ?>"><?php print $term_name; ?></a></h2>
The header is automatically generated. It provides information about which files are affected and what command was used to generate it (in this case the
git diff). The last two lines show the files being compared with a
--- (original) and
+++ (new) prefix.
diff --git a/modules/taxonomy/taxonomy-term.tpl.php b/modules/taxonomy/taxonomy-term.tpl.php index b515a9b..872d4cb 100644 --- a/modules/taxonomy/taxonomy-term.tpl.php +++ b/modules/taxonomy/taxonomy-term.tpl.php
Hunks of Changes
The first line of each hunk range represents the starting line number and the number of lines in the hunk. As above, the
+ prefixes refer to the original and the new version of the file(s). Below this, each change is represented with the original version of it, followed by the new version of it. The original version is prefixed with minus sign. It represents code being removed in the patch. The plus sign represents the new version of code being added.
@@ -37,7 +37,7 @@ * @see template_process() */ ?> -<div id="taxonomy-term-<?php print $term->tid; ?>" class="<?php print $classes; ?> clearfix"> +<div id="taxonomy-term-<?php print $term->tid; ?>" class="<?php print $classes; ?>"> <?php if (!$page): ?> <h2><a href="<?php print $term_url; ?>"><?php print $term_name; ?></a></h2>
Checking out a project from Drupal’s Git Repository with Tower
Before you can get started creating patches for Drupal projects, you’ll need to ensure you’ve got Git installed and that you’ve setup a local repository for the Drupal project you want to patch. You can test whether you’ve got Git installed by opening up terminal, typing “git” and hitting enter. If you get a bunch of help text, then you’re good. If you get a message saying the command isn’t found, then you’ll need to install Git.
-bash: git: command not found
If you are comfortable using the command line, this page will show you now to install Git. Afterwards, or if you’ve already got Git installed you can skip to creating a patch instructions. If you need a little guidance with a GUI, read on.
Set up a local repository with a GUI (Tower in this case)
Personally, I use and like Tower, so I’ll show you how to set up a repository using it. There are lots of Git GUI’s available and the process of setting up a new repository is pretty similar across them, so you’ll probably be able to follow along. Refer to the handbook for a growing list of Git GUI apps that are available.
Begin by click “Clone Remote Repository” from the dashboard.
Grab the repository URL from from the Version control tab (available from every project page) as use it as the “Remote URL.” In this case we are setting up the current development version of Drupal core. Name the repository whatever you want and browse to a location where you want to place the files. Then click “OK.”
After clicking “OK” the clone process will begin. It usually takes a minute or two depending on your connection speed. When it’s complete a new repository will appear on your dashboard. When you click into it you’ll see that repository is already on the proper branch (8.x) and there are no local changes pending. That’s it.
How to Create a Patch
The first thing we need is an issue to patch. Last month, mortendk noticed that an unwanted
.clearfix class had somehow made it into the
taxnomy-term.tpl.php template. Eventually, a new bug report against core was born. Morten didn’t know how to create a patch, but he tells us what should be done in the first comment. Unfortunately, if there is no actual patch, it will never reach “Reviewed and Tested by the Community” status and will not get fixed so that is just not enough. We need an actual patch. We know what the fix is here. We simply need to remove the “clearfix” class. So how do we do that?
Make sure the code is up to date
Since we’ve already setup the repository, the first thing we need to do is make sure our codebase is completely up to date. There should be no other changes pending in the local repository. If you are using Tower or another GUI, you’ll need to make sure any pending changes are reverted before continuing.
We can update the code in the Terminal with the following 2 commands:
git reset --hard git pull origin 8.x
In our GUI, we can click the “Pull” icon (in Tower it’s the second icon from the top left) to accomplish the same.
Make Your Changes and Save Them
Since we’re working with version control, we can freely make changes and then save those changes right in place. They can always be easily reverted, and since none of us actually have commit access to Drupal core (or most contributed projects), we cannot actually do any harm. That being the case, I’m going to go straight to the affected file:
modules/taxonomy/taxonomy-term.tpl.php. I’ve deleted the “clearfix” class and saved the file. Now when I look in Tower, it appears as “modified” and it’s got what looks like a patch underneath it.
Create the Patch
We are now ready to create our patch. Sorry folks, but we’re definitely going to have to use the command line for this one. Don’t worry — It’s 2 painless commands. I promise. First, we navigate to the root of the project and then we run the
git diffcommand to generate the patch.
cd ~/Dropbox/Sites/drupal-8 git diff > patch-name.patch
It gets more complicated when patches require adding, renaming and(or) removing files. In order to get these changes into a patch, Git needs to know about them. You cannot just
git addor “stage” in Tower and then run the patch command. This will not work. Instead, you should stage the files like you normally would for a commit. After doing this, you can generate the patch using the —staged option when generating the patch. For more information, see git diff documentation.
git diff --staged > patch-name.patch
This short video shows this process in action.
Upload the patch to Drupal.org
You’re done! Your new patch is sitting in the root of the project directory. Now it’s time to upload it. It’s always best to explain what the patch does in the comment to help reviewers. This one is really simple and so is the comment. The patch is automatically tested by the testbot to ensure that it doesn’t cause any unwanted test failures elsewhere. Generally markup and CSS patches don’t have an affect on unit tests, so most of the time these will pass. The green background indicates that this one has passed the tests. Yay!
If you read on in the issue, you’ll see that it is reviewed by and marked “reviewed and tested by the community” or what we like to call RTBC. This gets the attention of core committers. Just a few comments and weeks later, Dries committed the fix to both Drupal 7 and Drupal 8. SUCCESS! Now that wasn’t so hard was it? This is how it happens, folks.
How to Apply a Patch
The first thing we need to do is save the patch to our project root. One fancy trick to speed this process up is to use
wget. This command doesn’t come with OS X by default, so if you want it you’ll need to install it. Of course this is not required. “Save as” works just as well, but if you do have it this is the simple command you’d run from the project root to download the patch:
Next we apply the patch using a simple command, again, from the root of the project:
git apply -v taxonomy-clearfix-1198088-01.patch
Hint: You can start typing the first few letters of your patch name and use tab completion.
The -v flag stands for “verbose.” When using this, the command will provide feedback in Terminal showing how the patch has applied.
Once the patch has been applied you can begin testing it and making modifications to the code, if needed. You may easily create an updated version of the patch by repeating steps 3-4 in the How to create a patch section.
How to Revert a Patch
There are a few different ways to revert patches:
Revert changes for a specific file using a -R (revert) option:
git apply -R taxonomy-clearfix-1198088-01.patch
Revert a specific file using git checkout, or you can select the file and click the “revert” icon in your GUI.
git checkout modules/taxonomy/taxonomy-term.tpl.php
Reset the entire working tree with a hard reset:
git reset --hard
Here are some quick shortcuts to the documentation pages that detail everything you ever wanted to know about patches and more.