{"id":292,"date":"2021-01-11T18:24:07","date_gmt":"2021-01-11T17:24:07","guid":{"rendered":"https:\/\/nine30.info\/?p=292"},"modified":"2021-01-11T18:24:07","modified_gmt":"2021-01-11T17:24:07","slug":"auto-scale-application-with-vrealize-automation-and-vrealize-operations","status":"publish","type":"post","link":"https:\/\/nine30.nxt70.com\/index.php\/2021\/01\/11\/auto-scale-application-with-vrealize-automation-and-vrealize-operations\/","title":{"rendered":"Auto-scale Application with vRealize Automation and vRealize Operations"},"content":{"rendered":"\n<p>I was requested to implement an auto-scale use case for a web application. Personally, I am not a fan of this auto-scale use case for some reasons, but elaborating on this could be the topic for a nice 9:30 chat with my friend Luca and possibly the subject for a new blog post. This entry in our blog is about how to configure auto-scale with vRealize solution family. What I like of this use case is that it gave me the chance to experiment with something I really like (<strong><span style=\"color:#012f42\" class=\"tadv-color\">vRA REST API<\/span><\/strong>) and dip my toes with something I am less familiar (<span style=\"color:#012f42\" class=\"tadv-color\"><strong>vROps Alerts\/Policies and vRO Scriptable Tasks<\/strong><\/span>). For this specific lab I used version 8.2 of vRA\/vRO and vROps, but I am quite sure the same applies to 8.0 and 8.1 as well.<\/p>\n\n\n\n<p>The use case is about using vRA to deploy a web application that, in addition to other resources, presents one or more web server behind a load balancer. vROps monitors web server(s) load and if the load gets too higher for the current number of web servers it fires an alert that triggers a vRA deployment update day 2 action that increases the number of web servers. This is a scale-out scenario, <strong><span style=\"color:#012f42\" class=\"tadv-color\">a similar approach applies to the scale-in scenario<\/span><\/strong> that is triggered when load on web servers is lower than a given thresholds. Of course, in this second scenario the deployment update action reduces the number of web servers.<\/p>\n\n\n\n<p>This is a summary of the configurations I had to put together to implement the use case:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>A VMware Cloud Template that deploys a web app with a configuration suitable for scale out\/in (load balancer and a clustered frontend web servers);<\/li><li>A vROps alert definition with a recommendation that automatically runs an action (vRO Workflow) when the alert is triggered;<\/li><li>A vRO Workflow that executes a vRA ABX action (this is basically a wrapper for a vRA REST API request);<\/li><li>A vRA ABX action that determines the new size of the cluster and update the right deployment accordingly.<\/li><\/ol>\n\n\n\n<p>This could be implemented in many ways and certainly my approach is inefficient as points 3 and 4 in the list above could be implemented into a single vRO workflow.<\/p>\n\n\n\n<p>OK, let\u2019s get started.<\/p>\n\n\n\n<p>First things I needed was a <strong><span style=\"color:#012f42\" class=\"tadv-color\">VMware Cloud Template<\/span><\/strong> that deploys a web app with one or more web servers behind a load balancer. I am reusing a Cloud Template created by Chris McClanahan and his team, this template deploys a web app named <em>OpenCart<\/em> (an open source ecommerce platform <a href=\"https:\/\/www.opencart.com\/\">https:\/\/www.opencart.com\/<\/a>).<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"485\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/1-vra-ca-ctemplate.png\" alt=\"\" class=\"wp-image-485\"\/><\/figure>\n<\/figure>\n\n\n\n<p class=\"has-small-font-size\">(Click on the image to enlarge)<\/p>\n\n\n\n<p>Here the important point is that the template allows to have <strong><span style=\"color:#012f42\" class=\"tadv-color\">multiple VMs named <em>frontend<\/em> behind a NSX-T load balancer<\/span><\/strong>. We control the number of fronted VMs by means of an input parameters named <em>clusterSize<\/em> that can have the following values: small, medium and large.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>clusterSize:\n  type: string\n  enum:\n    - small\n    - medium\n    - large\n  default: small\n  title: Frontend Cluster Size\n  description: Frontend Cluster Size<\/code><\/pre>\n\n\n\n<p>Values of the <em>clusterSize<\/em> parameter are mapped as follows: small is 1 VM, medium 2 VMs and large 4 VMs.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>count: '${input.clusterSize == \"small\" ? 1 : (input.clusterSize == \"medium\" ? 2 : 4)}'<\/code><\/pre>\n\n\n\n<p>Next I had to create an <strong><span style=\"color:#012f42\" class=\"tadv-color\">ABX action on VRA<\/span><\/strong>, I didn\u2019t started from scratch as along with the Cloud Template Chris passed to me a python script that does 60% of the work I need. Here is the logic of final script that lays behind the ABX action:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>The action expects 2 inputs: <em>vrops_moid<\/em> is the MOID of the VM with the load too high\/low and the <em>direction<\/em> of the scale action: out (we need to add web servers) in (we need to remove web servers);<\/li><li>First thing the script retrieves the vRA deployment the <em>vrops_moid<\/em> belongs to;<\/li><li>It retrieves the current size of the frontend cluster;<\/li><li>It determines the new size of the cluster according to the current size and the <em>direction<\/em> input;<\/li><li>It updates the deployment with the new cluster size.<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-2 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"486\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/2-vra-abx-action.png\" alt=\"\" class=\"wp-image-486\"\/><\/figure>\n<\/figure>\n\n\n\n<p>Then I turned to vRO to build a simple vRO workflows that accepts a <em>VC:VirtualMachine<\/em> object as input, it authenticates on vRA to retrieve a bearer token and use it to invoke the vRA REST API that execute the ABX action we created before. To automatically run this workflow when a vROps alert is fired (auto-remediation of an alert according to the VMware terminology) the workflow is required to have only one input parameter and this input parameter has be of type&nbsp;<em>VC:&lt;VC datatype&gt;<\/em>. Actually, I created two vRO workflows named <em>AutoScaleOut<\/em> and <em>AutoScaleIn<\/em>, they are identical with the exception of the value assigned to the <em>direction<\/em> input parameter of the <em>AutoScale<\/em> ABX Action. As last step I created a package that includes workflows and their dependencies.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-3 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"505\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/3a-vro-workflow-autoscaleout-1.png\" alt=\"\" class=\"wp-image-505\"\/><\/figure>\n<\/figure>\n\n\n\n<p>As a final step I had to configure the vROps alert. In order to execute vRO workflow as alert auto remediation, it is required to have installed and configured the <strong><span style=\"color:#012f42\" class=\"tadv-color\">Management Pack for vRO<\/span><\/strong>. I am not covering this in this blog post. <a href=\"https:\/\/docs.vmware.com\/en\/Management-Packs-for-vRealize-Operations-Manager\/3.1.1\/vrealize-orchestrator\/GUID-2201CD50-71A5-4453-AB25-C7638DDDF54C.html\">Here<\/a> is the detailed procedure to configure and execute auto remediation. Hereafter I just go through the key steps I went through.<\/p>\n\n\n\n<p>First thing I had to import my vRO package into&nbsp;vROps. I had to be patient and wait for a collection cycle before my package including my workflows appeared along with those previously listed.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-4 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"489\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/4-vrops-config-package-discovery.png\" alt=\"\" class=\"wp-image-489\"\/><\/figure>\n<\/figure>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-5 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"506\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/3-vrops-import-vro-package.jpg\" alt=\"\" class=\"wp-image-506\"\/><\/figure>\n<\/figure>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-6 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"507\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/4-vrops-imported-vro-package.png\" alt=\"\" class=\"wp-image-507\"\/><\/figure>\n<\/figure>\n\n\n\n<p>Second, I mapped the my workflows to a&nbsp;vCenter&nbsp;resource of type&nbsp;<em>VC:VirtualMachine<\/em>&nbsp;(this have to match the&nbsp;vRO&nbsp;workflow input type).<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-7 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"508\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/4a-vrops-map-workflow.png\" alt=\"\" class=\"wp-image-508\"\/><\/figure>\n<\/figure>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-8 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"509\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/4b-vrops-mapped-workflow.jpg\" alt=\"\" class=\"wp-image-509\"\/><\/figure>\n<\/figure>\n\n\n\n<p>Then I turned my attention to create the custom alerts, but before that I created a new policy named <em>AutoScale<\/em> that applies only to <em>frontend<\/em> VMs (for this purpose I created a custom group that includes only frontend VMs). This policy was created to host my new custom alert definitions. I am pretty sure this is not the most appropriate approach, but it was just fine for my purpose.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-9 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"492\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/7-vrops-policy.png\" alt=\"\" class=\"wp-image-492\"\/><\/figure>\n<\/figure>\n\n\n\n<p>The remainder of this post just cover the creation of the custom alert definition for the scale out use case, the same procedure applies to the scale in alert definition with few minor differences (e.g. scale symptom threshold definition). First step to create my custom alert was to create a custom recommendation with action configured with the <em>AutoScaleOut<\/em> workflow.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-10 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"510\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/5-vrops-alert-recommandation.jpg\" alt=\"\" class=\"wp-image-510\"\/><\/figure>\n<\/figure>\n\n\n\n<p>Then I created a symptom which looks for VMs that have sustained CPU Usage (e.g. greater than or equal to 70%). I could add one or more symptoms related to Load Balancer Sessions or HTTP Requests (these and other metrics are available in vROps). This can be part of a monitoring tuning activity.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-11 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"511\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/6-vrops-alert-symptom-definition.jpg\" alt=\"\" class=\"wp-image-511\"\/><\/figure>\n<\/figure>\n\n\n\n<p>At this point I was ready to <strong><span style=\"color:#012f42\" class=\"tadv-color\">create my custom alert based on the symptom and recommendation created before<\/span><\/strong>. This alert will invoke the vRO workflow\/action automatically when CPU Usage of <em>frontend<\/em> Virtual Machine is higher or equal to 70%. Oh, by the way the new UI for alert definition and policy management is simply awesome and allows a person with little experience like me to get the job done quickly.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-12 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"512\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/8-vrops-alert-definition-symptom.png\" alt=\"\" class=\"wp-image-512\"\/><\/figure>\n<\/figure>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-13 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"513\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/9-vrops-alert-definition-recommendation.png\" alt=\"\" class=\"wp-image-513\"\/><\/figure>\n<\/figure>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-14 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"496\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/12-vrops-alert-definition-policy.png\" alt=\"\" class=\"wp-image-496\"\/><\/figure>\n<\/figure>\n\n\n\n<p>Finally I had to check that my alert had both State and Automate properties set to Enabled.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-15 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"514\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/11-vrops-policy-alert-automated.png\" alt=\"\" class=\"wp-image-514\"\/><\/figure>\n<\/figure>\n\n\n\n<p>Time to test it!<\/p>\n\n\n\n<p>I have a <em>OpenCart<\/em> Cloud Template based deployment with <em>clusterSize = small<\/em> (1 VM). On the only frontend VM I generated high CPU utilization long enough to trigger an alert, I used the <a href=\"https:\/\/linux.die.net\/man\/1\/stress\">stress<\/a> Linux tool.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-16 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"515\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/1-test-stress.png\" alt=\"\" class=\"wp-image-515\"\/><\/figure>\n<\/figure>\n\n\n\n<p>vROps triggered an alert according to my symptom definition<\/p>\n\n\n\n<div class=\"wp-block-jetpack-tiled-gallery aligncenter is-style-rectangular\"><div class=\"tiled-gallery__gallery\"><div class=\"tiled-gallery__row\"><div class=\"tiled-gallery__col\" style=\"flex-basis:100%\"><figure class=\"tiled-gallery__item\"><img decoding=\"async\" srcset=\"https:\/\/i2.wp.com\/nine30.info\/wp-content\/uploads\/2021\/01\/2-test-alert.jpg?strip=info&#038;w=600&#038;ssl=1 600w,https:\/\/i2.wp.com\/nine30.info\/wp-content\/uploads\/2021\/01\/2-test-alert.jpg?strip=info&#038;w=900&#038;ssl=1 900w,https:\/\/i2.wp.com\/nine30.info\/wp-content\/uploads\/2021\/01\/2-test-alert.jpg?strip=info&#038;w=1200&#038;ssl=1 1200w,https:\/\/i2.wp.com\/nine30.info\/wp-content\/uploads\/2021\/01\/2-test-alert.jpg?strip=info&#038;w=1500&#038;ssl=1 1500w,https:\/\/i2.wp.com\/nine30.info\/wp-content\/uploads\/2021\/01\/2-test-alert.jpg?strip=info&#038;w=1800&#038;ssl=1 1800w,https:\/\/i2.wp.com\/nine30.info\/wp-content\/uploads\/2021\/01\/2-test-alert.jpg?strip=info&#038;w=1920&#038;ssl=1 1920w\" alt=\"\" data-height=\"949\" data-id=\"516\" data-link=\"https:\/\/nine30.info\/auto-scale-application-with-vrealize-automation-and-vrealize-operations\/2-test-alert\/\" data-url=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/2-test-alert.jpg\" data-width=\"1920\" src=\"https:\/\/i2.wp.com\/nine30.info\/wp-content\/uploads\/2021\/01\/2-test-alert.jpg?ssl=1\" data-amp-layout=\"responsive\"\/><\/figure><\/div><\/div><\/div><\/div>\n\n\n\n<p>The alert automatically started the vRO workflow that through the ABX action triggered the deployment update.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-17 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"517\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/3-test-update-in-progress.png\" alt=\"\" class=\"wp-image-517\"\/><\/figure>\n<\/figure>\n\n\n\n<p>And finally my deployment ended up with 2 fronted VMs<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-18 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"518\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/01\/4-test-done.png\" alt=\"\" class=\"wp-image-518\"\/><\/figure>\n<\/figure>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Automation and operations help each other, applications&#8217; reliability is guaranteed<\/p>\n","protected":false},"author":1,"featured_media":407,"comment_status":"open","ping_status":"open","sticky":false,"template":"templates\/template-cover.php","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[4],"tags":[8,9,14,23,24,31,39,53,67,83,100,103,116,124,146,147,173,176,178,180,184,188,189,193],"class_list":["post-292","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech","tag-abx","tag-action","tag-api","tag-automation","tag-autoscaling","tag-bsx","tag-clustersize","tag-deployment","tag-frontend","tag-http-request","tag-load-balancer","tag-management-pack","tag-nsx-t","tag-operations","tag-scale-in","tag-scale-out","tag-virtual-machine","tag-vm","tag-vmware","tag-vmware-cloud-template","tag-vra","tag-vro","tag-vrops","tag-workflow","entry"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/posts\/292","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/comments?post=292"}],"version-history":[{"count":0,"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/posts\/292\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/"}],"wp:attachment":[{"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/media?parent=292"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/categories?post=292"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/tags?post=292"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}