The sacred product model

As with most System Integrators, we have been in the position to pick up failed Magento projects, some that have never launched and others that are launched but plagued with problems or are extremely difficult (some would say impossible) to maintain, upgrade, or enhance. If you are a seller and you are thinking about Magento this might alarm you and get your mind spinning. Why do these projects fail? How could I have a completed project that I now can't maintain, isn't that the whole point of using a platform? What should I look for to make sure this doesn't happen to me?

While not true in every case, often problems center around products ... how they were configured and what changes have been made to Magento to accommodate a customer's special product needs.

One of the most important decisions that must be made for a Magento system is to determine how to map a customers' products to the Magento product architecture. How the architect decides to represent products impacts all aspects of the project. Layers and layers of additional customizations can become necessary depending on the decision of how to handle products. Unfortunately, the details may seem too overwhelming to most customers and they simply trust that the development team is doing the right thing.

Sellers shouldn't be expected to fully understand the details, but they should be able to ask this basic question ... does the design change the way a product is represented in the Magento database? If the system integrator can't answer that question, they haven't done enough analysis. If the system integrator says yes ... throw up the red flag. Seek a full explanation. Get a second opinion. Press for details.

Why does this matter so much, why the emphatic response? The reason is that Magento product model is the heart and soul of the system. Everything (literally everything) is centered around it. The Magento world revolves around the product model sun. Here is a short sample list of areas impacted by changing the product model:

  • Tax Calculations
  • Shipping Calculations
  • Orders
  • Invoices
  • Shipments
  • Reporting
  • Search
  • Product Editing
  • Promotions
  • Pricing
  • Inventory Management
  • Product Rendering
  • On Category Pages
  • On Product Detail Pages
  • On Mini-Cart
  • On Cart Line Items

I will have some future posts documenting some product model change fails and clever (architecture-friendly) ways to leverage the product model.

The fallacy of extensions

Whoa! What are you talking about? Aren't extensions the cornerstone of Magento? Is this heresy? Well yes ... and no. There is a tremendous misunderstanding about the value and proper use of extensions. There tend to be several camps when it comes to extensions: haters, zealots, and pragmatists.

The Haters

The haters want no extensions. They want to limit themselves to the core Magento system whenever possible. We find many of the haters have previous Magento 1 experience and have many battle scars working through bugs generated by extensions, extension conflicts, and unexpected side-effects from unprecise extension coding. They are not wrong. Their pain is real. Extensions have a checkered past. System Integrators try to limit themselves to extensions they have used before and found them to already be working as expected or have already worked out the kinks with their own personalized version of them.

The problem of course is that core Magento rarely is not enough. The Magento brain trust has always understood this and that is why they have created and fostered the extension community. To reject all extensions is to give up some of the great power of the Magento system. Too often the result is that the customer may have to incur significant costs for custom development which may meet the exact need of "today" but rarely is built to be the flexible extensions that can adapt over time. This approach leads to a few lifecycle problems:

  • Enhancements are expensive because you are continuingly paying for custom development
  • Custom development often lacks the appropriate surrounding features like quality admins, logging, documentation
  • Customers never get to take advantage of the community of developers that are constantly improving and evolving their extensions.

In short, don't be a hater. You will have significantly more expensive projects, higher long term maintenance, and slower adoption of new features/improvements from the community

The Zealots

Extensions are the future. Who needs developers and engineering? I can find an extension for anything, install it myself, and be on my way. The zealots come from many paths but generally fall into a few categories: startups new to eCommerce, converts from old, legacy systems, and converts from pure custom solutions. These customers have read the marketing material and have bought the extension network hook, line, and sinker.

Anyone who has worked in Magento can tell you this is fool's gold. If you are looking to become a rescue project that a second, or third development team has to rescue from the dead, try to solve all of your problems with extensions.

Why is this? Is the extension community a lie? Have I been hoodwinked? How can I achieve my TCO (total cost of ownership) goals if I still need developers? I thought I was free of being held back by developers, you are telling me I am still beholden to them, how can this be?

The piece of pie was awesome, and the second one was pretty good, but the third one made me feel sick. Why? because too much of a good thing is not a good thing. Extensions are awesome when they do exactly what you need and nothing else. They are even more awesome when they are the only extension overriding some area of Magento. If you start piling the extensions you might become a Hater. All of a sudden that sweet extension for BOGO promotions is interfering with your special tiered pricing and free gift extensions and now the cart is broken. I have three extension vendors and no one thinks it is their problem ... This approach leads to a few problems:

  • Things are happening in your system that no one can explain
  • Your system is doing things in a way you don't really want but don't have the power to change
  • You are finding that the level of urgency you feel for a problem is not reciprocated by the extension owners because they are sure it is not them
  • You may not have a quality development team at your beck and call because you didn't think you needed them ...

Don't be a zealot either. Magento has a powerful framework and extensions can make life easier but nothing replaces a carefully crafted system with some intelligent design.

The Pragmatists

And then there are the pragmatists. They are cautious and selective about extensions but they realize the power extensions hold and they understand the delicate balance between initial cost, functionality, and long term maintainability. Every system integrator worth their salt and many of the happiest customers are pragmatists. They understand a successful Magento system still needs system design and an engineering team..

To truly get the system that you always wanted you need a strong dose of core Magento functionality, the right extensions, and carefully crafted custom code to tie it all together. If there is an extension that does exactly what you are looking for and fits nicely into its own niche it is a no-brainer. The right extension is dramatically cheaper for the customer, faster to implement, and will come with quality support from the extension provider. These extensions allow Magento to crush their competitors when it comes to implementation cost.

The pragmatist also knows that when a customer has multiple, specific needs around a portion of the Magento system that often the best solution is to build it yourself. Too often people make the foolish mistake of trying to couple multiple extensions together and write some "glue" code ... that seemingly never handles all of the scenarios and breaks every time any of the extensions need to be upgraded.

Be a pragmatist (I know it sounds very boring), find success.

Expect the unexpected

All programming is not created equal. There is application programming and there is framework programming for instance. Application programming tackles a specific set of problems toward a known solution. Framework programming considers the what if's and the "this should never happen but just in case".

In the world of programming, there are many application programmers and few framework programmers. This is natural. Framework programming takes a lot more thought and a lot more time up front. It is natural to lean towards application programming because customers can't appreciate the thoughtfulness of a better design or more robust solution as they are more bottom-line driven.

The dichotomy is that Magento application development is framework development. Magento core, extensions, and custom code must all work harmoniously together without explicit knowledge of each other. When code is written making assumptions about the system state or availability of objects with no consideration of the "what if" makes the process breakdown.

If you have done any extensive Magento work, you have seen this happen. Why does the enhanced analytics extension throw errors when I have this shipping extension enabled ... How could a chat extension cause address validation to fail ... and on and on.

The answer is simple ... the extensions make too many assumptions and have little to no fallback when the assumptions are not met. This is less true in Magento core but does still crop up from time to time. This is incredibly true with custom development.

We have inherited lots of code from different integrators so I can say with confidence that this is a common problem. I can't count how many times I have uttered the phrase "well they clearly didn't consider this scenario".

So how do you fix this problem ... what can be done?

The first and most important thing is to change the mindset. A developer can't look at a request and ONLY solve the very specific problem stated. They MUST consider the "what ifs". This is a cultural thing. From the top down, the entire engineering department must be on board.

Then as you develop ... you have to ask yourself questions about the code you are writing:

  1. How can my code gracefully fail?
  2. If the object(s) that I need to operate on aren't there ... what should my code do?
  3. What data am I assuming is available ... and what if it isn't?
  4. What are all of the scenarios that will happen that could cause my code to be run?

We recently wrote a pricing extension that was specifically for configurable products and it took multiple iterations to get it just right. Our original development fell short because we didn't fully consider all of the scenarios ...

  1. Do we handle every product type: simple, configurable, grouped, bundled?
  2. Have we covered every place that pricing is calculated? (add to cart versus cart updates)
  3. What happens if the product was added to the cart and then the underlying simple was disabled?

It takes more time on the front side to consider all of the scenarios but ultimately it takes no more time as you spend less time debugging issues on the back side ... and you don't introduce bugs into your customer's system.

Functionality vs Performance: The reorder conundrum

This post is part of a series of posts around achieving maintainability and scalability within your Magento solution. Magento offers a great feature that allows customers to reorder products previously purchased. In a vacuum the default solution makes perfect sense. Magento reviews each previous order and the items associated with it to determine if the items are still available in the catalog. If so, Magento presents a Reorder button next to the order in the order history grid.

Logically the solution makes sense, but is a very heavyweight solution that includes loading each product, all its variations, and corresponding stock statuses into memory to perform this check. The result is for a user that has purchased many items over time, the order history page is unimaginably slow for the user (sometimes to the point of time outs) and very resource intensive to the server.

It is important then to look at the profile of your customer and ask a few questions ...

  1. Do I have loyal customers who are making frequent purchases over time?
  2. How often is my product catalog turning over, will products from a year ago ever be re-orderable?
  3. How often do my customers come back to buy the same thing so that reorder even makes sense?

Every business is different, so you need to consider the factors above before deciding how valuable reorder is for you.

If you don't have many customers reordering, then this is a moot issue and you can move on. You can probably also stick with the base functionality if you do have customers that reorder but they generally reorder the same products over and over.

If you don't have customers ordering the same products over and/or your catalog turns over seasonally, reorder probably doesn't make sense. In this case, modifying Magento to remove the Reorder check and the Reorder button makes good sense.

If you do have reorders and the same customer may buy a wide range of items, you might want to consider limiting the reorder check for order history to the first few orders (5 or 10) and then disabling reorder checks for all over orders.

Teach them to fish

As my company has undertaken the journey that is Magento 2 we find ourselves constantly researching how others in the Magento community have solved similar problems. We are grateful for so many contributors willing to spend the time and effort to write up explanations of how they chose to solve problems and share with the rest us.

We are constantly searching for the proper balance between the quick solution that meets the customer's exact request (and current budget) and the fundamentally sound solution that takes longer and maybe is more than the customer really wanted to tackle and way more then they wanted to spend. Much like in Magento 1 there are often lots of ways to accomplish a single task in Magento 2. Unlike Magento 1, a developer can carefully craft a change that is injected at just the right time in the request life cycle with limited impact to the rest of the system. Unfortunately, it is still too easy for a developer to make changes that have terrible side effects as what they believed was a small change for some specific scenario actually was a generic function that is called by lots of different processes with data in all sorts of different states.

So how do we collectively learn? .. how do we learn to customize Magento better? ... how do we impart the wisdom we learn to others? ... It is my hope that this blog can play a part in this process. So much of the intellectual property on Magento is recanting a rote implementation for a specific problem. This sort of help can be exactly what you need ... or not. What if what you need is sorta like this other developers problem but not quite ... Did the other developer approach the problem correctly? Does the solution align well with the core Magento architecture? Is the solution one that is made to last? These are harder questions ... but just as important as solving the problem at hand.

As many other Magento developers can attest to when they take over projects build by other teams, there are a LOT of bad implementations. It is difficult to tell a customer that while their system "looks" right from the front-end perspective, the underlying implementation is a disaster ... that there is significant work to be able to upgrade their Magento version due to the customizations put in place ... that core Magento features don't work because the previous team violated the core Magento framework assumptions ... that the previous team gave them a solution that was not maintainable.

We are not perfect, we make mistakes, we make bad design decisions, but we don't want to. We want every customer to have a system that is built to last with thoughtful design and core Magento architecture principles maintained. This blog is part of our effort to explain more than just the how ... but the why when executing custom Magento development.

We don't want to just give fish to eat, we want to teach people how to fish ...