{"id":991,"date":"2021-08-09T11:04:22","date_gmt":"2021-08-09T10:04:22","guid":{"rendered":"https:\/\/nine30.info\/?p=991"},"modified":"2021-08-09T11:04:22","modified_gmt":"2021-08-09T10:04:22","slug":"vra-saltstack-config-configure-git-fileserver","status":"publish","type":"post","link":"https:\/\/nine30.nxt70.com\/index.php\/2021\/08\/09\/vra-saltstack-config-configure-git-fileserver\/","title":{"rendered":"vRA SaltStack Config &#8211; Configure git fileserver"},"content":{"rendered":"\n<p>Salt relies on files (e.g. State files, Reactor config files, Pillars, etc.), for this reason it comes with a simple <strong>file server<\/strong> suitable for distributing files to the Salt Minions. The file server is a stateless ZeroMQ server that comes with the Salt Master. The main goal of the Salt file server is to present files for use in the Salt state system.<\/p>\n\n\n\n<p>Salt Master supports different <strong>file server backends<\/strong>. File server backends allow the Salt file server to act as a transparent bridge to external resources. A good example of this is the&nbsp;<code>git<\/code>&nbsp;backend, which allows Salt to serve files sourced from one or more git repositories, but there are several others as well.&nbsp;<a rel=\"noreferrer noopener\" href=\"https:\/\/docs.saltproject.io\/en\/latest\/ref\/file_server\/all\/index.html#all-salt-fileserver\" target=\"_blank\">Here<\/a>&nbsp;you can access the full list of Salt&#8217;s file server backends.<\/p>\n\n\n\n<p>In this post we will configure Salt Master to integrate with git backend, specifically I tested this procedure with GitHub, but it is applicable to any git. Before jumping into the process it is good to mention that the git file server backend is named <code>gitfs<\/code> and it can be enabled by adding&nbsp;<code>git<\/code>fs&nbsp;to the&nbsp;<code><a rel=\"noreferrer noopener\" href=\"https:\/\/docs.saltproject.io\/en\/latest\/ref\/configuration\/master.html#std-conf_master-fileserver_backend\" target=\"_blank\">fileserver_backend<\/a><\/code>&nbsp;list and configuring one or more repositories in the&nbsp;<a href=\"https:\/\/docs.saltproject.io\/en\/latest\/ref\/configuration\/master.html#std-conf_master-gitfs_remotes\"><code>gitfs_remotes<\/code><\/a> parameter in a Salt Master configuration file. Branches and tags become Salt fileserver environments. A more experienced Salt user suggested me to start with a simple repository without branches and use different repos for prod and dev Salt instances. I am transferring this piece of wisdom to you even though I not really following it in my lab.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Install Dependencies<\/h2>\n\n\n\n<p>Here we assume you have a working instance of Salt Master and thus you have Python and <a rel=\"noreferrer noopener\" href=\"http:\/\/www.pip-installer.org\/\" target=\"_blank\">Pip<\/a> on your system, so just make sure they are updated. The procedure in this post is tested for vRA SaltStack Config 8.4 (installed via vRLCM) running on PhotonOS 3.0.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"1009\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/06\/01.prerequisites.png\" alt=\"\" class=\"wp-image-1009\"\/><\/figure>\n<\/figure>\n\n\n\n<p>Both&nbsp;<a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/libgit2\/pygit2\" target=\"_blank\">pygit2<\/a>&nbsp;and&nbsp;<a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/gitpython-developers\/GitPython\" target=\"_blank\">GitPython<\/a>&nbsp;are supported Python interfaces to git. If compatible versions of both are installed,&nbsp;pygit2 is preferred. In this scenario,&nbsp;GitPython can be forced using the&nbsp;<a rel=\"noreferrer noopener\" href=\"https:\/\/docs.saltproject.io\/en\/latest\/ref\/configuration\/master.html#std-conf_master-gitfs_provider\" target=\"_blank\"><code>gitfs_provider<\/code><\/a>&nbsp;parameter in the master config file (we cover this later in the &#8220;Configure the git backend&#8221; section). A longtime Salt user suggested me to use GitPython and I am following his advice here. <\/p>\n\n\n\n<p>GitPython requires the&nbsp;<code>git&nbsp;CLI<\/code> utility to work. If GitPython is installed from a system package, then git should already be installed, but if installed via&nbsp;Pip then it may still be necessary to install git separately.<\/p>\n\n\n\n<p>Install git:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>yum install git<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-2 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"1010\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/06\/03.git_.install.png\" alt=\"\" class=\"wp-image-1010\"\/><\/figure>\n<\/figure>\n\n\n\n<p>Install GitPython:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>pip3 install gitpython<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-3 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"1011\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/06\/02.gitpython.install.png\" alt=\"\" class=\"wp-image-1011\"\/><\/figure>\n<\/figure>\n\n\n\n<p>If you have any issue related to GitPython you can refer to <a rel=\"noreferrer noopener\" href=\"https:\/\/docs.saltproject.io\/en\/latest\/topics\/tutorials\/gitfs.html\" target=\"_blank\">this<\/a> walkthrough.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Configure the git backend<\/h2>\n\n\n\n<p>The very minimum configuration to start using git backend requires two parameters (<code>fileserver_backend<\/code> and <code>gitfs_remotes<\/code>) to be set in a master configuration file. Rather than adding these parameters in the <code>\/etc\/salt\/master.d\/raas.conf<\/code> file that comes with the vRLCM installation, it is a good idea to create a new configuration file dedicated to the <code>gitfs<\/code> backend, we just need to make sure the new file name comes before &#8220;raas.conf&#8221; in the alphabetical order. This ensures that git backend configurations are picked from our new configuration file and any other git backend configuration specified in other file(s) (having a name that comes after in the alphabetical order ) is ignored.<\/p>\n\n\n\n<p>Create a new configuration file, in my case I named it <code>gitfs.conf<\/code>:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>vi \/etc\/salt\/master.d\/gitfs.conf<\/code><\/pre>\n\n\n\n<p>Add the following configurations:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>fileserver_backend:\n  - sseapi\n  - roots\n  - gitfs\n\ngitfs_provider: gitpython\n\ngitfs_interval: 60\n\ngitfs_base: main\n\ngitfs_root: \/\n\ngitfs_mountpoint: \/\n\ngitfs_remotes:\n - &lt;your git repo URL&gt;<\/code><\/pre>\n\n\n\n<p><a rel=\"noreferrer noopener\" href=\"https:\/\/docs.saltproject.io\/en\/latest\/ref\/configuration\/master.html#std-conf_master-gitfs_remotes\" target=\"_blank\">Here<\/a> is the documentation for all the supported <code>gitfs<\/code> parameters, in this post I&#8217;ll just focus on the two mandatory parameters and the gitfs_provider:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><a rel=\"noreferrer noopener\" href=\"https:\/\/docs.saltproject.io\/en\/latest\/ref\/configuration\/master.html#std-conf_master-fileserver_backend\" target=\"_blank\"><code>fileserver_backend<\/code><\/a>: this let you list all the supported backend in your Salt Master, in my case I am using <code>sseapi<\/code>, <code>roots<\/code> and of course <code>gitfs<\/code>;<\/li><li><a rel=\"noreferrer noopener\" href=\"https:\/\/docs.saltproject.io\/en\/latest\/ref\/configuration\/master.html#std-conf_master-gitfs_remotes\" target=\"_blank\"><code>gitfs_remote<\/code><\/a>: this lets you specify Specify one or more&nbsp;<code>git:\/\/<\/code>,&nbsp;<code>https:\/\/<\/code>,&nbsp; <code>file:\/\/<\/code> or&nbsp;<code>ssh:\/\/<\/code>&nbsp;URLs to configure which repositories to cache and search for requested files. The user running the Salt Master will need read access to the repo. The repos will be searched in order to find the file requested by a client and the first repo to have the file will return it;<\/li><li><a rel=\"noreferrer noopener\" href=\"https:\/\/docs.saltproject.io\/en\/latest\/ref\/configuration\/master.html#gitfs-provider\" target=\"_blank\">gitfs_provider<\/a>: this is an optional parameter used to specify the provider to be used for gitfs. It must be either&nbsp;<code>pygit2<\/code>&nbsp;or&nbsp;<code>gitpython<\/code>. If unset, then each will be tried in that same order, and the first one with a compatible version installed will be the provider that is used. I strongly recommend to configure <code>gitfs_provider<\/code>.<\/li><\/ul>\n\n\n\n<p>When configuring <code>gitfs_remote<\/code> you just provide a repo URL (with your State Files) using one of the supported protocols with the following syntax:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>HTTP: <code>https:\/\/github.com\/username\/repositoryname.git<\/code><\/li><li>GIT: <code>git@github.com:username\/repo.git<\/code><\/li><li>SSH: <code>ssh:\/\/username@domain.tld\/path\/to\/repo.git<\/code><\/li><\/ul>\n\n\n\n<p>Restart the Salt Master:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>systemctl restart salt-master<\/code><\/pre>\n\n\n\n<p>Keep in mind that if your repository is not public, then there are some other authentication pieces that need to be in place as well. <a rel=\"noreferrer noopener\" href=\"https:\/\/docs.saltproject.io\/en\/latest\/ref\/configuration\/master.html#gitfs-authentication-options\" target=\"_blank\">Here<\/a> you can access configuration parameters documentation for the various supported authentication options, while <a rel=\"noreferrer noopener\" href=\"https:\/\/docs.saltproject.io\/en\/latest\/topics\/tutorials\/gitfs.html#gitfs-authentication\" target=\"_blank\">here<\/a> you can access the authentication section of an excellent <code>gitfs<\/code> walkthrough .<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Check<\/h2>\n\n\n\n<p>On your Salt Master you can list directories available in all your backend servers and make sure the folders of your git repository (assuming you have folders in you repo) are there:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>salt-run fileserver.dir_list<\/code><\/pre>\n\n\n\n<p>List environments in all your backend servers and make sure envs match with you repository branches (there might be more envs defined in other backends). Please, note that by default the <code>main\/master<\/code> branch of a repository is mapped on the Salt <code>base<\/code> env. You can use the <code>gitfs_base<\/code> parameter in the master configuration file to defines which branch\/tag should be used as the&nbsp;<code>base<\/code>&nbsp;environment:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>salt-run fileserver.envs<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-4 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"1006\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/06\/04.gitfs_.check_.png\" alt=\"\" class=\"wp-image-1006\"\/><\/figure>\n<\/figure>\n\n\n\n<p>List files, in my case I am listing files in the <code>dev<\/code> environment that is the <code>dev<\/code> branch of my repository:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>salt-run fileserver.file_list saltenv=dev<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-5 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"1007\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/06\/05.gitfs_.check_.2.png\" alt=\"\" class=\"wp-image-1007\"\/><\/figure>\n<\/figure>\n\n\n\n<p>Finally apply a State File, in my case I am using the <code>test=True<\/code> option just to make sure I do not screw up things:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>salt saltmaster state.apply reactor\/master\/master test=True<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-6 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"1008\" src=\"https:\/\/nine30.info\/wp-content\/uploads\/2021\/06\/06.test_.png\" alt=\"\" class=\"wp-image-1008\"\/><\/figure>\n<\/figure>\n\n\n\n<p>That&#8217;s it! Now you can work with Salt using git. Since you are here, I also suggest to use VS code to edit your Salt Files, there is a nice SaltStack extension that adds language colorization support for the SaltStack template language and supports yaml with Jinja2 templating.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Salt relies on files (e.g. State files, Reactor config files, Pillars, etc.), for this reason it comes with a simple file server suitable for distributing files to the Salt Minions. The file server is a stateless ZeroMQ server that comes with the Salt Master. The main goal of the Salt file server is to present&hellip; <a class=\"more-link\" href=\"https:\/\/nine30.nxt70.com\/index.php\/2021\/08\/09\/vra-saltstack-config-configure-git-fileserver\/\">Continue reading <span class=\"screen-reader-text\">vRA SaltStack Config &#8211; Configure git fileserver<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":569,"comment_status":"open","ping_status":"open","sticky":false,"template":"templates\/template-cover.php","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[4],"tags":[71,72,73,99,122,126,141,143,144,145,178,185,186,190],"class_list":["post-991","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech","tag-git","tag-github","tag-gitops","tag-linux","tag-opensource","tag-oss","tag-python","tag-salt","tag-saltproject","tag-saltstack","tag-vmware","tag-vrealize","tag-vrealizeautomation","tag-vscode","entry"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/posts\/991","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/comments?post=991"}],"version-history":[{"count":0,"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/posts\/991\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/"}],"wp:attachment":[{"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/media?parent=991"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/categories?post=991"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nine30.nxt70.com\/index.php\/wp-json\/wp\/v2\/tags?post=991"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}