Blog

How to Import Products into Magento for your Enterprise System

Product Management Workflow

Complete eCommere solutions often involve numerous enterprise systems working together. Product management can be one of the most labor intensive and error prone aspects of maintaining your solution.

Clients generally default to three common solutions for handling product management:

  1. Add/Update/Delete products using the Magento Commerce product administration screens.
  2. Upload or add new products using the Magento product import tool.
  3. Installing a rudimentary data upload extension like uRapidflow or Xtento.

If your organization lacks complexity, your product updates are infrequent, or you have simple product configurations these solutions may work for you. For the vast majority of customers we encounter this is not the case. All the above solutions, lack any customer-specific business intelligence on how to organize products (e.g. auto-generation of parent configurable based on sku format, grouping of products into bundles based on particular criteria, etc.)

If you are already using Magento and find your team:

  • Publishing incomplete products
  • Pushing products with errors
  • Manipulating data in spreadsheets
  • Uploading countless different spreadsheets
  • Passing spreadsheets around between team members
  • Having difficulties detecting incomplete products or inadvertently breaking products on updates

… there is a better way.

Some customers can achieve complete automation and never use product management within Magento while most others use a combination of automation and precise workflow to efficiently handle products. A customer's precise capabilities and needs should be reviewed to determine the best architecture for them.

Full automation is only possible if the product data source has complete information capable of properly merchandising (e.g. descriptions, images), pricing, and categorizing a product. Additionally, well defined rules must be established on how new attributes, new attribute values, and new categories are handled.

In these scenarios, the ERP/PIM/Product Data Provider owns the product data and is responsible for product additions and updates. These systems often have limited or fixed formats for data export and are not easily updated. Magento is responsible for ingesting the product feed on a regular basis and performing the following functions:

  • Mapping product feed data to Magento taxonomy
  • Auto generation of new attributes
  • Auto generation of new attribute values
  • Auto configuration/reconfiguration of the category hierarchy
  • Auto generation of new products
  • Auto updating of existing products
  • Auto archiving of deleted products

Depending on the nature of customer products and customer needs, some of the automations may not be necessary. Category structures, attributes, and accepted attribute values may all be maintained directly in Magento.

More often the source of product data does not contain the necessary information to properly sell a product online. In these scenarios, there is joint ownership of product data between systems. The external system feeding product data controls the fields they know and understand and Magento controls and owns the fields necessary to complete the setup of the product.

In these scenarios, a combination of automation and workflow makes sense. Magento ingests the feeds and checks for product existence. If the product already exists, the data controlled by the third-party source is automatically updated. If the product is new, the feed data is mapped to Magento data and the product is dropped into a customer-specific workflow to complete the product setup.

The workflow varies by customer based on their organizational setup. The simplest contains a single role but can easily be split among a number of specific functions (e.g. copywriter, image processor, content manger, approver).

If you are interested in learning more about Kadro’s enhancements to Magento to allow to easier product uploading contact us at sales@kadro.com or visit our Contact Page and fill out our online form.

12 Things to Consider When Moving to Magento Commerce Cloud

Magento Commerce Cloud

Considering moving from an on-premise solution, or alternate cloud hosting solution, to Magento Commerce Cloud?

The great news is the core Magento code does not change. It is possible your Magento solution can be moved into Magento Commerce Cloud with little difficulty. Here are some things to evaluate when considering the move:

1. File System Limitations

The file system on Commerce Cloud is read-only except for the pub/media and var folders. If you have any extensions or custom code that is relying on file system access it is imperative that the code uses the Magento-provided functions rather than pure PHP or PHP framework methods. Attempting to use the non-Magento functions will result in file permission issues.

This is commonly seen in file transfer processes between Magento and 3rd party systems or the generation of log/activity files.

2. Fastly Cache Integration

If your Magento solution is not already integrated with Fastly for CDN and caching, there are a few things to consider. Any code that defines blocks that are not cacheable will cause Fastly to not cache entire pages. This is considered bad practice by Magento. If a block isn't cacheable, the block should be delivered after initial page load via an AJAX request back to the server. Occasionally we see this in custom code, but more often see this in poorly executed third party extensions.

Fastly does provide a tool for you to verify that the caching is working correctly on your site.

Your current CDN solution will no longer be necessary.

3. 301 Redirects

Commerce Cloud only provides limited redirect functionality by default. Simple redirect rules can be applied to your corresponding YAML file. If your current solution has pattern-based redirects (e.g a legacy system using request parameters to define pages) you will need to reimplement these redirects as VCL snippets to install into Fastly.

4. File System Access

Commerce Cloud requires that server access runs through SSH key exchanges. If you have current integrations or business processes that presumes unfettered access to the server or basic login/password credentials, they will need to be upgraded to use SSH keys. Any processes that are leveraging file copies or basic ftp will need to be revised to instead use sftp to transfer files to and from the Commerce Cloud server.

In the event that your local processes cannot accommodate the more secure ssh key exchange, you will need to revise your Magento installation to pull and push from the Magento side rather from your internal application side.

5. Database Access

Commerce Cloud only allows direct access to the database instance from the application server. To access the database you will need to use your registered SSH key and use SSH tunneling. Tools like DbForge will allow you to setup the SSH tunneling and access that database like you would in normal circumstances.

Database administration can be accomplished through a tool that supports SSH tunneling or by ssh'ing to the application server and using the MySQL cli interface.

6. WordPress

Numerous non-cloud installations still use WordPress for blog content. This is not recommended by Magento due to the security vulnerabilities associated with WordPress and is forbidden within the Commerce Cloud setup. Only in rare circumstances (e.g. you have a team of WordPress experts, use many WordPress extensions, and have significant content separate from the e-commerce solution) would maintaining WordPress make sense.

There are several blog extensions that provide popular blog features and serve as a reasonable replacement for WordPress. If your solution must continue to use WordPress you may need to reorganize your site to ensure WordPress will be installed in a separate server infrastructure apart from Magento Commerce.

7. Domain-based Licensed Extensions

Some extensions limit their use by domain names. This licensing structure is incompatible with Commerce Cloud. Commerce Cloud allows customers to spin up new environments with a few button clicks. Each of these environments will have a new domain name. At a minimum your site will have four domains (production, staging, integration, and local development). If your site has multiple stores and the stores use unique domains, each additional store will be another 4 domains at a minimum.

8. Realtime Integrations

If your application is relying on real-time communications to your ERP or OMS system, you may have been leveraging the fact that your system was within the same LAN (local area network) as Magento. With Magento moved to the Commerce Cloud there will be significant latency issues compare to a LAN connection that may make your solution untenable. You may need to work with the cloud infrastructure team to make special provisions for traffic between your server environment and the cloud environment.

9. Active WAF

The default setup for Commerce Cloud only includes a passive WAF from Fastly. If you have suffered through some DDOS attacks and are using active WAF's from companies like Cloudflare you have to consider a few options:

a. Subscribe to the Fastly Active WAF in addition to the default options.

b. Setup a double proxy that routes Cloudflare (or like) -> Fastly -> Magento (not recommended by Magento).

c. Forgo your current active WAF setup and work within the passive WAF capabilities

10. Bulk Data Loading

Commerce Cloud does not allow for bulk inserts into the MySQL database using the PHP MySQL interface. If you are bulk inserting data, you will need to configure the YAML file to specifically add the bulk operation permission to the database user.

11. Composer-based Deployments

Commerce Cloud uses GIT and composer to execute deployments. If you have made modifications to the vendor folder (and you should never do this), these changes will be lost on deployment to cloud. The deployed code in the vendor folder will be completely determined by the composer.lock file.

If you have hot fixes from Magento, they will need to be placed in the m2-hotfixes folder. When placed there, the hot fixes will be auto-installed on each deployment.

If you have been disabling extensions by editing the config.php file, you will need to modify a deployment file to make sure that they remained disabled after a deployment. Direct editing of the config.php on the cloud post deployment will not work.

12. Local Development Environments

Lastly, a common misunderstanding for customers new to Commerce Cloud is that the integration environment is where development takes place. Developers should be running the system on their own local environments to take advantage of modern development tools like PHP Storm.

If you don't already have local development environments, plan to spend some time setting up those environments. This includes a separate GIT repository for tracking all code changes within the standard GITFlow process. Commerce Cloud GIT is not your source code repository, rather it is simply the mechanism for pushing application builds to an environment.

October 2019 Newsletter: Celebrate the Success

Enjoy wine by the beach

"I do believe in praising that which deserves to be praised." -- Dean Smith

As we get ready for another great holiday season it's almost apropos that we have launched a first at Kadro, Duplin Winery. Duplin is our first wine and beverage customer we have launched on Magento Commerce, and they are truly some of the best folks we have had the opportunity to work with. You can read more about the project below, but visit their new site and order a case or two for the holidays!

Latest Project

Duplin Winery

The Magento Commerce project for Duplin Winery included several key integrations that are mission critical for the wine and beverage industry. The biggest being ShipCompliant by SOVOS to handle all of their compliance needs. Read more about this exciting project.

Featured Technology Partner

dotdigital Logo

dotdigital is a global leader in customer engagement technology and Engagement Cloud is its platform. With features that empower marketers to acquire, convert, and retain customers at scale. Users can connect with CRM and ecommerce systems, surface powerful data, and automate intelligent cross-channel messages that resonate with audiences. Learn more

Move to Magento

Kadro is a Magento Enterprise Solution Partner

Are you ready to transition to Magento eCommerce? Let Kadro help you!

Marketplace Manager

Kadro is a Magento Enterprise Solution Partner

bring your brand to new channels with our affordable omni-channel tool backed by the highly experienced e-commerce team at Kadro.

10 Tips for Complying with Sales Tax Regulations

10 Tips for Complying with Sales Tax Regulations

Repost from Avalara

There’s nothing sexy about sales tax compliance. There really isn’t. But there is something mighty compelling about not running afoul of the tax authorities.

 

Following some simple tips can help put even the least assiduous person on the path to successful sales tax compliance. And so, without further ado, here they are:

 

1.  Confirm your customers’ tax-exempt status. Exemption certificates need to be accurate, up to date, and readily accessible.

2.  Don’t ignore tax notices. Nothing good ever comes from letting a tax notice molder at the bottom of your to-do stack, even if you’re sure you’re blameless.

3.  Efile when you can (and when you must). Some states require electronic filing for all businesses, while others prefer it. For more information about a particular state’s requirements, check out the Avalara TaxRates state guides.

4.  Find out if you’re required to make prepayments. If you are, your filing requirements may differ from the standard filing requirements. For example, returns for some prepayments must be filed online in North Carolina, while others must be filed on paper.

5.  Get to know state nexus rules. If you have sales tax nexus with a state, you’re required to register with the tax department and collect and remit sales tax. Nexus rules have been changing at a rapid rate since the Supreme Court of the United States removed the physical presence rule in June 2018; it’s now easier than ever for a state to impose a sales tax collection obligation on remote sellers. See this state-by-state guide to sales tax nexus for up-to-date information.

6.  Identify taxability rules in states where you collect. Taxability rules for products and services differ from state to state (even from city to city in some states). Be sure you’re collecting what you should wherever you have nexus.

7.  Keep excellent records. You’ll need them if you’re ever audited. Auditors need to be able to track and verify each transaction from start to finish.

8.  Update filing frequencies on your tax calendars. A change in your business can lead to a change in filing status. It’s up to you to know how frequently you’re required to file.

9.  Reconcile! Reconcile! Reconcile! Start with the account balance at the beginning of the accounting period, then add the amount billed to customers and subtract the sales and use tax paid. The end result should match the current balance of your sales tax payable account.

10.   Use geolocation to determine sales tax rates. Relying on ZIP codes can lead to inaccurate sales tax rates. Geolocation provides more accurate rates, down to the rooftop.

Finally, consider automating sales tax compliance. Sales tax compliance is both essential and thankless: Your customers won’t praise you for getting sales tax right, but they’ll be sure let you know if you get it wrong. Automating sales tax calculation, collection, and/or remittance helps minimize costs and increase your chances of getting it right.

Want to know more? Get the updated Sales & Use Tax Compliance for Dummies ebook


Magento Commerce Cloud Demands Your Attention

Magento Commerce Cloud Demands Your Attention

I have heard the noise from other system integrators;

  • Magento Commerce Cloud is hard to work with
  • We prefer our technology stack
  • My system engineering team wants more control
  • The Magento Commerce Cloud support team is not responsive
  • The Magento Commerce Cloud is too immature
  • and on and on…….

I must admit it, I do not get it. The single greatest achievement for Magento 2 Commerce is the introduction of the cloud infrastructure. I am serious. Yes, the code architecture is vastly improved and the growth in content management is fantastic. Magento 2 is across the board a vast improvement over Magento 1. Yet, none of it compares to Magento Commerce Cloud.

So why all the push back?

Why are some resisting?

Why would anyone not recommend Commerce Cloud versus self-hosted?

Yes there are times a customer has unique requirements that requires on-premise hosting. Other times customers want their own IT staff to manage their servers. Those customers will never use Commerce Cloud. Everyone else should be using Commerce Cloud.

When the choice is up to us on what to recommend to a customer, we choose Commerce Cloud every time. No question. All we have to do is look at our Magento customer base and document the hosting related issues we have encountered in the last two years to understand anything besides Cloud is simply not the best choice.

Hosting and environment problems stink. There is always urgency. There is always finger pointing. No-one is happy. Customers push back on those large bills to address hosting emergencies. Anyone who has been responsible for an e-commerce site has experienced this pain. Here's the proof in the proverbial pudding:

15% of Magento clients are not on Commerce Cloud. They represent 90% of the hosting issues we have to deal with!

The problems are numerous and span the entire hosting spectrum. Here are just a few:

  • Auto-OS updates breaking Magento
  • Physical hardware failure
  • Poor caching solutions
  • ISP security breaches
  • Poor system monitoring
  • Insecure system configurations

Some hosting providers don't understand Magento at all and others have "experts". Unfortunately, those experts are rarely involved until you have gone through multiple tiers of support and a problem remains unresolved. Our customers still call us first when there is a problem. They may be paying for hosting company support, but they rely on us because we are more responsive and are capable of determining if the issue is hosting or application related. For us, the hosting companies simply don't provide much value. We spend much of our time explaining to the providers why what they are suggesting doesn't even make sense (they almost always want to upgrade hardware/software before they really understand what the root cause of the issue is).

Before you think that my system engineering team just stinks, a number of these issues are from inherited projects from other system integrators that are clearly not following Magento's best practices. That tells me one of the very things that Commerce Cloud was created to address over Magento 1 is still happening in Magento 2. Outside of the Commerce Cloud, Magento SIs are still getting it wrong.

Commerce Cloud is of course not perfect. No hosting is. We have experienced a few issues and had some miscommunications. Yet we are on the same team, Kadro and Magento, working together for our joint customer and have worked it out every single time. When Magento published that they had no system downtime during the holiday season, that should have been a wake up call for the doubters. I know we experienced NO system issues during the holidays from any of our clients on Magento Cloud, NONE!. It was amazing. To be honest I was in a bit of disbelief ... but how wonderful. This is the future and the future is now.