A photo of Malcolm Allen

Malcolm Allen

Malcolm is responsible for leading the overall technical direction of the company, including the development of software architectures and frameworks as well as working with customers on designing and developing their solutions. He has over 20 years experience in software engineering applied in the areas of product development and professional services consulting. He has extensive experience in developing enterprise-level web applications, integrating mobile solutions with web applications, database design, and distributed computing using a variety of technologies and architectures.

Should You Build Your Own B2B Marketplace Podcast


Business buyers are shopping on Amazon more than ever before. Today a growing number of B2B businesses are building their own marketplaces - and expanding their market share. If you want to be a leader and grow faster, it may be time to launch a marketplace of your own. Join Shannon Hane, senior product marketing manager at Adobe, and Malcolm Allen, Kadro's CTO, for an enlightening podcast. 


How to Import Products into Magento for your Enterprise System

Product Management Workflow

Complete eCommere solutions often involve numerous enterprise systems working together. Product management can be one of the most labor intensive and error prone aspects of maintaining your solution.

Clients generally default to three common solutions for handling product management:

  1. Add/Update/Delete products using the Magento Commerce product administration screens.
  2. Upload or add new products using the Magento product import tool.
  3. Installing a rudimentary data upload extension like uRapidflow or Xtento.

If your organization lacks complexity, your product updates are infrequent, or you have simple product configurations these solutions may work for you. For the vast majority of customers we encounter this is not the case. All the above solutions, lack any customer-specific business intelligence on how to organize products (e.g. auto-generation of parent configurable based on sku format, grouping of products into bundles based on particular criteria, etc.)

If you are already using Magento and find your team:

  • Publishing incomplete products
  • Pushing products with errors
  • Manipulating data in spreadsheets
  • Uploading countless different spreadsheets
  • Passing spreadsheets around between team members
  • Having difficulties detecting incomplete products or inadvertently breaking products on updates

… there is a better way.

Some customers can achieve complete automation and never use product management within Magento while most others use a combination of automation and precise workflow to efficiently handle products. A customer's precise capabilities and needs should be reviewed to determine the best architecture for them.

Full automation is only possible if the product data source has complete information capable of properly merchandising (e.g. descriptions, images), pricing, and categorizing a product. Additionally, well defined rules must be established on how new attributes, new attribute values, and new categories are handled.

In these scenarios, the ERP/PIM/Product Data Provider owns the product data and is responsible for product additions and updates. These systems often have limited or fixed formats for data export and are not easily updated. Magento is responsible for ingesting the product feed on a regular basis and performing the following functions:

  • Mapping product feed data to Magento taxonomy
  • Auto generation of new attributes
  • Auto generation of new attribute values
  • Auto configuration/reconfiguration of the category hierarchy
  • Auto generation of new products
  • Auto updating of existing products
  • Auto archiving of deleted products

Depending on the nature of customer products and customer needs, some of the automations may not be necessary. Category structures, attributes, and accepted attribute values may all be maintained directly in Magento.

More often the source of product data does not contain the necessary information to properly sell a product online. In these scenarios, there is joint ownership of product data between systems. The external system feeding product data controls the fields they know and understand and Magento controls and owns the fields necessary to complete the setup of the product.

In these scenarios, a combination of automation and workflow makes sense. Magento ingests the feeds and checks for product existence. If the product already exists, the data controlled by the third-party source is automatically updated. If the product is new, the feed data is mapped to Magento data and the product is dropped into a customer-specific workflow to complete the product setup.

The workflow varies by customer based on their organizational setup. The simplest contains a single role but can easily be split among a number of specific functions (e.g. copywriter, image processor, content manger, approver).

If you are interested in learning more about Kadro’s enhancements to Magento to allow to easier product uploading contact us at sales@kadro.com or visit our Contact Page and fill out our online form.

12 Things to Consider When Moving to Magento Commerce Cloud

Magento Commerce Cloud

Considering moving from an on-premise solution, or alternate cloud hosting solution, to Magento Commerce Cloud?

The great news is the core Magento code does not change. It is possible your Magento solution can be moved into Magento Commerce Cloud with little difficulty. Here are some things to evaluate when considering the move:

1. File System Limitations

The file system on Commerce Cloud is read-only except for the pub/media and var folders. If you have any extensions or custom code that is relying on file system access it is imperative that the code uses the Magento-provided functions rather than pure PHP or PHP framework methods. Attempting to use the non-Magento functions will result in file permission issues.

This is commonly seen in file transfer processes between Magento and 3rd party systems or the generation of log/activity files.

2. Fastly Cache Integration

If your Magento solution is not already integrated with Fastly for CDN and caching, there are a few things to consider. Any code that defines blocks that are not cacheable will cause Fastly to not cache entire pages. This is considered bad practice by Magento. If a block isn't cacheable, the block should be delivered after initial page load via an AJAX request back to the server. Occasionally we see this in custom code, but more often see this in poorly executed third party extensions.

Fastly does provide a tool for you to verify that the caching is working correctly on your site.

Your current CDN solution will no longer be necessary.

3. 301 Redirects

Commerce Cloud only provides limited redirect functionality by default. Simple redirect rules can be applied to your corresponding YAML file. If your current solution has pattern-based redirects (e.g a legacy system using request parameters to define pages) you will need to reimplement these redirects as VCL snippets to install into Fastly.

4. File System Access

Commerce Cloud requires that server access runs through SSH key exchanges. If you have current integrations or business processes that presumes unfettered access to the server or basic login/password credentials, they will need to be upgraded to use SSH keys. Any processes that are leveraging file copies or basic ftp will need to be revised to instead use sftp to transfer files to and from the Commerce Cloud server.

In the event that your local processes cannot accommodate the more secure ssh key exchange, you will need to revise your Magento installation to pull and push from the Magento side rather from your internal application side.

5. Database Access

Commerce Cloud only allows direct access to the database instance from the application server. To access the database you will need to use your registered SSH key and use SSH tunneling. Tools like DbForge will allow you to setup the SSH tunneling and access that database like you would in normal circumstances.

Database administration can be accomplished through a tool that supports SSH tunneling or by ssh'ing to the application server and using the MySQL cli interface.

6. WordPress

Numerous non-cloud installations still use WordPress for blog content. This is not recommended by Magento due to the security vulnerabilities associated with WordPress and is forbidden within the Commerce Cloud setup. Only in rare circumstances (e.g. you have a team of WordPress experts, use many WordPress extensions, and have significant content separate from the e-commerce solution) would maintaining WordPress make sense.

There are several blog extensions that provide popular blog features and serve as a reasonable replacement for WordPress. If your solution must continue to use WordPress you may need to reorganize your site to ensure WordPress will be installed in a separate server infrastructure apart from Magento Commerce.

7. Domain-based Licensed Extensions

Some extensions limit their use by domain names. This licensing structure is incompatible with Commerce Cloud. Commerce Cloud allows customers to spin up new environments with a few button clicks. Each of these environments will have a new domain name. At a minimum your site will have four domains (production, staging, integration, and local development). If your site has multiple stores and the stores use unique domains, each additional store will be another 4 domains at a minimum.

8. Realtime Integrations

If your application is relying on real-time communications to your ERP or OMS system, you may have been leveraging the fact that your system was within the same LAN (local area network) as Magento. With Magento moved to the Commerce Cloud there will be significant latency issues compare to a LAN connection that may make your solution untenable. You may need to work with the cloud infrastructure team to make special provisions for traffic between your server environment and the cloud environment.

9. Active WAF

The default setup for Commerce Cloud only includes a passive WAF from Fastly. If you have suffered through some DDOS attacks and are using active WAF's from companies like Cloudflare you have to consider a few options:

a. Subscribe to the Fastly Active WAF in addition to the default options.

b. Setup a double proxy that routes Cloudflare (or like) -> Fastly -> Magento (not recommended by Magento).

c. Forgo your current active WAF setup and work within the passive WAF capabilities

10. Bulk Data Loading

Commerce Cloud does not allow for bulk inserts into the MySQL database using the PHP MySQL interface. If you are bulk inserting data, you will need to configure the YAML file to specifically add the bulk operation permission to the database user.

11. Composer-based Deployments

Commerce Cloud uses GIT and composer to execute deployments. If you have made modifications to the vendor folder (and you should never do this), these changes will be lost on deployment to cloud. The deployed code in the vendor folder will be completely determined by the composer.lock file.

If you have hot fixes from Magento, they will need to be placed in the m2-hotfixes folder. When placed there, the hot fixes will be auto-installed on each deployment.

If you have been disabling extensions by editing the config.php file, you will need to modify a deployment file to make sure that they remained disabled after a deployment. Direct editing of the config.php on the cloud post deployment will not work.

12. Local Development Environments

Lastly, a common misunderstanding for customers new to Commerce Cloud is that the integration environment is where development takes place. Developers should be running the system on their own local environments to take advantage of modern development tools like PHP Storm.

If you don't already have local development environments, plan to spend some time setting up those environments. This includes a separate GIT repository for tracking all code changes within the standard GITFlow process. Commerce Cloud GIT is not your source code repository, rather it is simply the mechanism for pushing application builds to an environment.

Magento Commerce Cloud Demands Your Attention

Magento Commerce Cloud Demands Your Attention

I have heard the noise from other system integrators;

  • Magento Commerce Cloud is hard to work with
  • We prefer our technology stack
  • My system engineering team wants more control
  • The Magento Commerce Cloud support team is not responsive
  • The Magento Commerce Cloud is too immature
  • and on and on…….

I must admit it, I do not get it. The single greatest achievement for Magento 2 Commerce is the introduction of the cloud infrastructure. I am serious. Yes, the code architecture is vastly improved and the growth in content management is fantastic. Magento 2 is across the board a vast improvement over Magento 1. Yet, none of it compares to Magento Commerce Cloud.

So why all the push back?

Why are some resisting?

Why would anyone not recommend Commerce Cloud versus self-hosted?

Yes there are times a customer has unique requirements that requires on-premise hosting. Other times customers want their own IT staff to manage their servers. Those customers will never use Commerce Cloud. Everyone else should be using Commerce Cloud.

When the choice is up to us on what to recommend to a customer, we choose Commerce Cloud every time. No question. All we have to do is look at our Magento customer base and document the hosting related issues we have encountered in the last two years to understand anything besides Cloud is simply not the best choice.

Hosting and environment problems stink. There is always urgency. There is always finger pointing. No-one is happy. Customers push back on those large bills to address hosting emergencies. Anyone who has been responsible for an e-commerce site has experienced this pain. Here's the proof in the proverbial pudding:

15% of Magento clients are not on Commerce Cloud. They represent 90% of the hosting issues we have to deal with!

The problems are numerous and span the entire hosting spectrum. Here are just a few:

  • Auto-OS updates breaking Magento
  • Physical hardware failure
  • Poor caching solutions
  • ISP security breaches
  • Poor system monitoring
  • Insecure system configurations

Some hosting providers don't understand Magento at all and others have "experts". Unfortunately, those experts are rarely involved until you have gone through multiple tiers of support and a problem remains unresolved. Our customers still call us first when there is a problem. They may be paying for hosting company support, but they rely on us because we are more responsive and are capable of determining if the issue is hosting or application related. For us, the hosting companies simply don't provide much value. We spend much of our time explaining to the providers why what they are suggesting doesn't even make sense (they almost always want to upgrade hardware/software before they really understand what the root cause of the issue is).

Before you think that my system engineering team just stinks, a number of these issues are from inherited projects from other system integrators that are clearly not following Magento's best practices. That tells me one of the very things that Commerce Cloud was created to address over Magento 1 is still happening in Magento 2. Outside of the Commerce Cloud, Magento SIs are still getting it wrong.

Commerce Cloud is of course not perfect. No hosting is. We have experienced a few issues and had some miscommunications. Yet we are on the same team, Kadro and Magento, working together for our joint customer and have worked it out every single time. When Magento published that they had no system downtime during the holiday season, that should have been a wake up call for the doubters. I know we experienced NO system issues during the holidays from any of our clients on Magento Cloud, NONE!. It was amazing. To be honest I was in a bit of disbelief ... but how wonderful. This is the future and the future is now.

10+ Reasons Why Magento Commerce Cloud is the Solution for You

10+ Reasons Why Magento Commerce Cloud is the Solution for You

Why is Magento Commerce Cloud the right solution for you? Here a few reasons why we believe Magento Commerce Cloud is the right solution for hosting your Magento solution.

  1. Magento Commitment to Site Excellence

    Magento's reputation is on the line, they know it and act accordingly. Magento has demonstrated to us that they are committed to ensuring that customers' sites perform well. The commitment extends well beyond the initial configuration. Magento monitors your system to:

    1. Identify potential performance issues on live systems.
    2. Monitor system usage to ensure your site stays within acceptable performance metrics.
    3. If for some reason your site experiences slowness, they proactively reach out and are available to help diagnose the problem.
  2. Guaranteed Right-sized Provisioning

    Something we have never experienced before. Magento Commerce Cloud won't allow you to deploy your site to an undersized environment. Magento Commerce Cloud won't configure an environment that is not sufficient. Your key site metrics (peak page views, transactions, concurrent users, concurrent admin users, etc.) are factored into the resources you will need to have a performant system. Hosting companies ask you want you want, Magento Commerce Cloud educates you what you need.

  3. Knowledgeable Hosting Staff

    Unlike any other hosting provider, the Commerce Cloud team know and understand the complete Magento environment infrastructure. They are experts in configuring Magento. They know and have implemented Magento best practices. Your site will not be a snowflake, it will follow the proven server architecture to achieve the best performance for your site.

  4. Staging Environment that matches production

    Each Commerce Cloud setup includes a production and a staging environment. The staging environment runs the same services and software that the production environment does. This allows site changes to be fully vetted before pushing to production.

  5. Simplicity of Server Access

    Getting access to the servers is simple. No special VPN clients required. Add a user through the web portal and assign them a role. Developers and support staff need only an SSH key and they can be accessing environments in minutes.

  6. Docker instances for development

    In addition to have a staging environment that mirrors production for quality assurance and user acceptance testing, Magento provides a docker setup so that your development team can also develop in the same environment as the code will eventually be deployed in, eliminating unfound errors during deployment because of mismatched environments.

  7. Ease of environment replication

    Magento Commerce Cloud makes it very simple to copy an upper environment down into a lower environment or to spin up a new environment based on an existing one. Want to do some testing with the latest production data, no problem. Want to give a 3rd-party developer access to their own environment to track down an issue with their code or service, no problem.

  8. Up-sizing of environments

    Part of the Magento Commerce Cloud package is a set number of days that Magento will upsize your environment during peak usage periods. You can even have Magento auto-upsize your environment when they determine that resources are being maxed out. Whether it be for the holiday season at the end of the calendar year, on specific catalog drops throughout the year, or the end of the government's fiscal year, your environment can be increased to handle the server load you know is coming.

  9. Blackfire, Fastly, New Relic

    Magento Commerce Cloud bundles in a rich suite of services to help ensure your site reaches its optimal performance. Fastly manages your SSL certificates, your web application firewall, and your CDN. New Relic provides in-depth performance analysis of the entire application stack. Blackfire allows you to debug site issues directly on production without negatively impacting your running site.

  10. Build Tools

    Magento Commerce Cloud makes pushing new builds to a server environment easy. A simple GIT push installs new code and initiates a complete rebuild of the environment.

  11. Best Practice Configurations

    Sites on Magento Commerce Cloud follow best practices for performance and security.

  12. Continued Process Improvement

    Magento Commerce Cloud is fairly new. The tools and configurations available today have improved since initial launch. Magento is committed to continuing to improve what is already the best solution for customers.

Leveraging Magento Commerce Cloud to Grow My Business

Leveraging Magento Commerce Cloud to Grow My Business

As a Magento System Integrator we work with clients through the entire life cycle of their site: project inception, site build, ongoing site maintenance and so forth. Our roots have always been in software development. It is what we love. Our core team of software engineers is an elite group of e-commerce and web development experts.

When we first forayed into Magento seven years ago we didn't fully appreciate the hosting nightmare we were walking to. There seemed to be no standards on how a Magento site was hosted. We started our practice by inheriting existing Magento sites as many system integrators do. What was amazing is that every site we encountered was its own snowflake. There were so many possibilities and it seemed that everyone was trying a bit of everything without clear direction.

There were different flavors of UNIX, different branches of PHP, different web servers, different MySQL variations, different caching strategies, and on and on. We investigated a number of hosting providers that claimed to have just the right configuration for Magento sites, except we also inherited clients on those same providers that were struggling with performance and security. We also read a number of published white papers on Magento hosting providing contradictory recommendations.

We realized that we needed to figure this out for ourselves. So while hosting had always been an afterthought for our business it was now front and center. We grew a systems engineering team, hired new employees and transitioned other engineers into this effort. We documented all of our clients, and their unique configurations. We began to establish our own "solution" to hosting Magento.

We then found ourselves in the same place our clients did focussing on efforts that were not our core business. Magento merchants find system integrators because software development and maintenance is not their core business. Clients want to focus on the essence of their business, not the technology to make it happen. Here we were in a very precarious spot, a software development company now being burdened with hosting, unix environments, server management and collating a set of services/programs to run a Magento sites.

In comes the savior Magento Commerce Cloud. A solution provided by Magento to fully manage environments, security, hosting and content delivery. A true hosting solution implemented with all of the best practices, leveraging powerful integrated services and run by the people that know Magento best - Magento!.

Custom build scripts for each customer ... gone.

Server setup and maintenance ... gone.

Picking and testing the right solution set (web server, caching, CDN) ... gone.

Managing unique security requirements for clients ... gone.

Convincing clients of the importance of aligned production and staging instances ... gone.

The pain of setting up additional environments for testing ... gone.

For us, no longer did we need all of the system engineers. Our software developers that were constantly distracted by system configurations, they are back to developing.

With Magento Commerce Cloud we are back to being who we are ... software developers. The increased focus has allowed us to do more and be more for our clients. It is no coincidence the rise of Magento Commerce Cloud parallels the growth of my company.

8 Keys to Success on Magento Commerce Cloud

Magento Commerce Cloud

Magento Commerce Cloud has become the favored Magento configuration in our organization. I thought I would share some things that have been the key to our success.

  1. Embrace the infrastructure

    Engineers are opinionated. System engineers are no different. They like the way they do it. They have passion. We had to convince our system engineers to acquiesce to the will of the Magento team. There was some initial hesitation and push back, but we decided to align with the Magento Commerce Cloud's approach. It might have been a little rough at first but ultimately it has paid great dividends for our organization. Things we used to have to manage, stress over, and care about now are afterthoughts. If you embrace it, you can leverage it to your benefit and to your customer's benefit.

  2. Embrace Magento

    This might sound strange. You are already using Magento so of course you have embraced Magento. We have been involved with taking over a few Magento Commerce Cloud implementations that were struggling and can tell some are not truly embracing Magento.

    • Magento has a package loader and you should use it.
    • Magento has a patching process and you should use it.
    • Magento is tested and optimized for LESS and you should use it.
    • Magento Commerce Cloud has a solution for 301 redirects and you should use it.
    • Magento has a solution for JavaScript loading and you should use it.

    Every time you stray from what Magento has, your team has to shift focus from solving business problems to managing development, build, and environment issues.

  3. Do development elsewhere

    I have seen a number of other companies try to use the integration environment for development. This does not work. The integration environment does not have the horse power to do proper development. There are good uses for that environment but development is not one of them.

    Our team has their own virtual machines and/or docker setups for customers. We do our development from our office with the use of a shared database server. Depending on what each team member is doing we may have just one or multiple databases setup for the customer. Each developer has their own application server configured with XDebug and uses PHPStorm to write and debug code within their instance.

    We follow the GitFlow process and have individual developers add code updates into feature branches and an integration prime responsible for reviewing, merging, and deploying code to the cloud for QA testing, UAT, and production builds.

  4. Use default processes

    The cloud build process allows for developers to add in their own build hooks to adjust the process. From the installations we have reviewed, it seems some development teams try to morph the process to align with their own internal deployment process. Those installations fight against the tide. Each time the Magento Commerce Cloud team modifies, fixes, or improves the build process, these custom modifications fail. It requires that the development team keep doing their own testing on each cloud deployment release rather than actually working on development for their customers.

    The default processes work. They are simple. They will meet your needs.
  5. Leverage the tool set

    Magento Commerce Cloud bundles in some nice technology and features that we regularly take advantage of. New Relic is a great tool to monitor the performance of your Magento site. Especially with complex builds, since there are a lot of integrations and custom programming, New Relic quickly shows what process(es) is consuming the most resources to help us isolate the problem. Where New Relic helps with the what, Blackfire.IO helps with the why. Blackfire allows you to connect to your running production server and debug an individual session. We have used this effectively to debug the exact problems found through New Relic.

    The cloud tools make it simple to spin up a new instance into the Integration environment. While this environment doesn't work for development, it does work well for giving access to an extension provider to debug and address issues with their code in an isolated environment without impacting any other ongoing development and testing. Add in your own script for scrubbing data clean and the normal hesitation you get when giving access to a 3rd party is mitigated.

  6. Use composer sparingly

    This might surprise some. Isn't composer the wave of the future? Don't you want to automatically get the latest updates from extension builders? Don't you want the last code from your System Integrator's shared custom extensions? In short, nope. You can require an exact version in composer so no unexpected updates happen, but then on every build you are dependent on 3rd party servers to be running and responsive.

    Magento 2 has excellent code separation making it very easy to manage extensions. You never want to get automatic extension updates. The system should be fully verified in your development process and approved through staging. This entails knowing EXACTLY what code is in use. If between builds you pull in unexpected new code, your system is no longer fully tested and unexpected problems crop up.

  7. Staging is for staging

    As already mentioned, the integration environment is not for development. That shouldn't be interpreted as a reason to use the staging environment for development. Staging is for staging. Staging uses the same configuration (multi-server, Fastly, etc.) as production so is a perfect place for final QA and UAT prior to releasing to production. There should be no surprises when publishing to production. If staging is used for staging, there won't be. Worry free production releases are possible.

  8. Just because you can doesn't mean you should

    Magento has built in a lot of flexibility to allow developers to divest from the Magento default direction in not only code, but tools and utilities. Magento has such a wide audience and enormous open-source community this makes sense. Magento Commerce however is built for the enterprise. Enterprise solutions are already complicated. Magento Commerce Cloud provides the hosting and infrastructure for an enterprise solution. Just use it.

The sacred product model

As with most System Integrators, we have been in the position to pick up failed Magento projects, some that have never launched and others that are launched but plagued with problems or are extremely difficult (some would say impossible) to maintain, upgrade, or enhance. If you are a seller and you are thinking about Magento this might alarm you and get your mind spinning. Why do these projects fail? How could I have a completed project that I now can't maintain, isn't that the whole point of using a platform? What should I look for to make sure this doesn't happen to me?

While not true in every case, often problems center around products ... how they were configured and what changes have been made to Magento to accommodate a customer's special product needs.

One of the most important decisions that must be made for a Magento system is to determine how to map a customers' products to the Magento product architecture. How the architect decides to represent products impacts all aspects of the project. Layers and layers of additional customizations can become necessary depending on the decision of how to handle products. Unfortunately, the details may seem too overwhelming to most customers and they simply trust that the development team is doing the right thing.

Sellers shouldn't be expected to fully understand the details, but they should be able to ask this basic question ... does the design change the way a product is represented in the Magento database? If the system integrator can't answer that question, they haven't done enough analysis. If the system integrator says yes ... throw up the red flag. Seek a full explanation. Get a second opinion. Press for details.

Why does this matter so much, why the emphatic response? The reason is that Magento product model is the heart and soul of the system. Everything (literally everything) is centered around it. The Magento world revolves around the product model sun. Here is a short sample list of areas impacted by changing the product model:

  • Tax Calculations
  • Shipping Calculations
  • Orders
  • Invoices
  • Shipments
  • Reporting
  • Search
  • Product Editing
  • Promotions
  • Pricing
  • Inventory Management
  • Product Rendering
  • On Category Pages
  • On Product Detail Pages
  • On Mini-Cart
  • On Cart Line Items

I will have some future posts documenting some product model change fails and clever (architecture-friendly) ways to leverage the product model.

The fallacy of extensions

Whoa! What are you talking about? Aren't extensions the cornerstone of Magento? Is this heresy? Well yes ... and no. There is a tremendous misunderstanding about the value and proper use of extensions. There tend to be several camps when it comes to extensions: haters, zealots, and pragmatists.

The Haters

The haters want no extensions. They want to limit themselves to the core Magento system whenever possible. We find many of the haters have previous Magento 1 experience and have many battle scars working through bugs generated by extensions, extension conflicts, and unexpected side-effects from unprecise extension coding. They are not wrong. Their pain is real. Extensions have a checkered past. System Integrators try to limit themselves to extensions they have used before and found them to already be working as expected or have already worked out the kinks with their own personalized version of them.

The problem of course is that core Magento rarely is not enough. The Magento brain trust has always understood this and that is why they have created and fostered the extension community. To reject all extensions is to give up some of the great power of the Magento system. Too often the result is that the customer may have to incur significant costs for custom development which may meet the exact need of "today" but rarely is built to be the flexible extensions that can adapt over time. This approach leads to a few lifecycle problems:

  • Enhancements are expensive because you are continuingly paying for custom development
  • Custom development often lacks the appropriate surrounding features like quality admins, logging, documentation
  • Customers never get to take advantage of the community of developers that are constantly improving and evolving their extensions.

In short, don't be a hater. You will have significantly more expensive projects, higher long term maintenance, and slower adoption of new features/improvements from the community

The Zealots

Extensions are the future. Who needs developers and engineering? I can find an extension for anything, install it myself, and be on my way. The zealots come from many paths but generally fall into a few categories: startups new to eCommerce, converts from old, legacy systems, and converts from pure custom solutions. These customers have read the marketing material and have bought the extension network hook, line, and sinker.

Anyone who has worked in Magento can tell you this is fool's gold. If you are looking to become a rescue project that a second, or third development team has to rescue from the dead, try to solve all of your problems with extensions.

Why is this? Is the extension community a lie? Have I been hoodwinked? How can I achieve my TCO (total cost of ownership) goals if I still need developers? I thought I was free of being held back by developers, you are telling me I am still beholden to them, how can this be?

The piece of pie was awesome, and the second one was pretty good, but the third one made me feel sick. Why? because too much of a good thing is not a good thing. Extensions are awesome when they do exactly what you need and nothing else. They are even more awesome when they are the only extension overriding some area of Magento. If you start piling the extensions you might become a Hater. All of a sudden that sweet extension for BOGO promotions is interfering with your special tiered pricing and free gift extensions and now the cart is broken. I have three extension vendors and no one thinks it is their problem ... This approach leads to a few problems:

  • Things are happening in your system that no one can explain
  • Your system is doing things in a way you don't really want but don't have the power to change
  • You are finding that the level of urgency you feel for a problem is not reciprocated by the extension owners because they are sure it is not them
  • You may not have a quality development team at your beck and call because you didn't think you needed them ...

Don't be a zealot either. Magento has a powerful framework and extensions can make life easier but nothing replaces a carefully crafted system with some intelligent design.

The Pragmatists

And then there are the pragmatists. They are cautious and selective about extensions but they realize the power extensions hold and they understand the delicate balance between initial cost, functionality, and long term maintainability. Every system integrator worth their salt and many of the happiest customers are pragmatists. They understand a successful Magento system still needs system design and an engineering team..

To truly get the system that you always wanted you need a strong dose of core Magento functionality, the right extensions, and carefully crafted custom code to tie it all together. If there is an extension that does exactly what you are looking for and fits nicely into its own niche it is a no-brainer. The right extension is dramatically cheaper for the customer, faster to implement, and will come with quality support from the extension provider. These extensions allow Magento to crush their competitors when it comes to implementation cost.

The pragmatist also knows that when a customer has multiple, specific needs around a portion of the Magento system that often the best solution is to build it yourself. Too often people make the foolish mistake of trying to couple multiple extensions together and write some "glue" code ... that seemingly never handles all of the scenarios and breaks every time any of the extensions need to be upgraded.

Be a pragmatist (I know it sounds very boring), find success.

Expect the unexpected

All programming is not created equal. There is application programming and there is framework programming for instance. Application programming tackles a specific set of problems toward a known solution. Framework programming considers the what if's and the "this should never happen but just in case".

In the world of programming, there are many application programmers and few framework programmers. This is natural. Framework programming takes a lot more thought and a lot more time up front. It is natural to lean towards application programming because customers can't appreciate the thoughtfulness of a better design or more robust solution as they are more bottom-line driven.

The dichotomy is that Magento application development is framework development. Magento core, extensions, and custom code must all work harmoniously together without explicit knowledge of each other. When code is written making assumptions about the system state or availability of objects with no consideration of the "what if" makes the process breakdown.

If you have done any extensive Magento work, you have seen this happen. Why does the enhanced analytics extension throw errors when I have this shipping extension enabled ... How could a chat extension cause address validation to fail ... and on and on.

The answer is simple ... the extensions make too many assumptions and have little to no fallback when the assumptions are not met. This is less true in Magento core but does still crop up from time to time. This is incredibly true with custom development.

We have inherited lots of code from different integrators so I can say with confidence that this is a common problem. I can't count how many times I have uttered the phrase "well they clearly didn't consider this scenario".

So how do you fix this problem ... what can be done?

The first and most important thing is to change the mindset. A developer can't look at a request and ONLY solve the very specific problem stated. They MUST consider the "what ifs". This is a cultural thing. From the top down, the entire engineering department must be on board.

Then as you develop ... you have to ask yourself questions about the code you are writing:

  1. How can my code gracefully fail?
  2. If the object(s) that I need to operate on aren't there ... what should my code do?
  3. What data am I assuming is available ... and what if it isn't?
  4. What are all of the scenarios that will happen that could cause my code to be run?

We recently wrote a pricing extension that was specifically for configurable products and it took multiple iterations to get it just right. Our original development fell short because we didn't fully consider all of the scenarios ...

  1. Do we handle every product type: simple, configurable, grouped, bundled?
  2. Have we covered every place that pricing is calculated? (add to cart versus cart updates)
  3. What happens if the product was added to the cart and then the underlying simple was disabled?

It takes more time on the front side to consider all of the scenarios but ultimately it takes no more time as you spend less time debugging issues on the back side ... and you don't introduce bugs into your customer's system.

Functionality vs Performance: The reorder conundrum

This post is part of a series of posts around achieving maintainability and scalability within your Magento solution. Magento offers a great feature that allows customers to reorder products previously purchased. In a vacuum the default solution makes perfect sense. Magento reviews each previous order and the items associated with it to determine if the items are still available in the catalog. If so, Magento presents a Reorder button next to the order in the order history grid.

Logically the solution makes sense, but is a very heavyweight solution that includes loading each product, all its variations, and corresponding stock statuses into memory to perform this check. The result is for a user that has purchased many items over time, the order history page is unimaginably slow for the user (sometimes to the point of time outs) and very resource intensive to the server.

It is important then to look at the profile of your customer and ask a few questions ...

  1. Do I have loyal customers who are making frequent purchases over time?
  2. How often is my product catalog turning over, will products from a year ago ever be re-orderable?
  3. How often do my customers come back to buy the same thing so that reorder even makes sense?

Every business is different, so you need to consider the factors above before deciding how valuable reorder is for you.

If you don't have many customers reordering, then this is a moot issue and you can move on. You can probably also stick with the base functionality if you do have customers that reorder but they generally reorder the same products over and over.

If you don't have customers ordering the same products over and/or your catalog turns over seasonally, reorder probably doesn't make sense. In this case, modifying Magento to remove the Reorder check and the Reorder button makes good sense.

If you do have reorders and the same customer may buy a wide range of items, you might want to consider limiting the reorder check for order history to the first few orders (5 or 10) and then disabling reorder checks for all over orders.

Teach them to fish

As my company has undertaken the journey that is Magento 2 we find ourselves constantly researching how others in the Magento community have solved similar problems. We are grateful for so many contributors willing to spend the time and effort to write up explanations of how they chose to solve problems and share with the rest us.

We are constantly searching for the proper balance between the quick solution that meets the customer's exact request (and current budget) and the fundamentally sound solution that takes longer and maybe is more than the customer really wanted to tackle and way more then they wanted to spend. Much like in Magento 1 there are often lots of ways to accomplish a single task in Magento 2. Unlike Magento 1, a developer can carefully craft a change that is injected at just the right time in the request life cycle with limited impact to the rest of the system. Unfortunately, it is still too easy for a developer to make changes that have terrible side effects as what they believed was a small change for some specific scenario actually was a generic function that is called by lots of different processes with data in all sorts of different states.

So how do we collectively learn? .. how do we learn to customize Magento better? ... how do we impart the wisdom we learn to others? ... It is my hope that this blog can play a part in this process. So much of the intellectual property on Magento is recanting a rote implementation for a specific problem. This sort of help can be exactly what you need ... or not. What if what you need is sorta like this other developers problem but not quite ... Did the other developer approach the problem correctly? Does the solution align well with the core Magento architecture? Is the solution one that is made to last? These are harder questions ... but just as important as solving the problem at hand.

As many other Magento developers can attest to when they take over projects build by other teams, there are a LOT of bad implementations. It is difficult to tell a customer that while their system "looks" right from the front-end perspective, the underlying implementation is a disaster ... that there is significant work to be able to upgrade their Magento version due to the customizations put in place ... that core Magento features don't work because the previous team violated the core Magento framework assumptions ... that the previous team gave them a solution that was not maintainable.

We are not perfect, we make mistakes, we make bad design decisions, but we don't want to. We want every customer to have a system that is built to last with thoughtful design and core Magento architecture principles maintained. This blog is part of our effort to explain more than just the how ... but the why when executing custom Magento development.

We don't want to just give fish to eat, we want to teach people how to fish ...