{"id":254,"date":"2013-03-10T15:01:19","date_gmt":"2013-03-10T22:01:19","guid":{"rendered":"http:\/\/www.ossintegrators.com\/blog\/?p=254"},"modified":"2013-03-10T15:10:21","modified_gmt":"2013-03-10T22:10:21","slug":"splunk-revision-control-subversion-example","status":"publish","type":"post","link":"http:\/\/www.ossintegrators.com\/blog\/splunk-revision-control-subversion-example\/","title":{"rendered":"Splunk + Revision Control (Subversion Example)"},"content":{"rendered":"<div style=\"padding-bottom:20px; padding-top:10px;\" class=\"hupso-share-buttons\"><!-- Hupso Share Buttons - http:\/\/www.hupso.com\/share\/ --><a class=\"hupso_toolbar\" href=\"http:\/\/www.hupso.com\/share\/\"><img decoding=\"async\" src=\"http:\/\/static.hupso.com\/share\/buttons\/share-small.png\" border=\"0\" style=\"padding-top:5px; float:left;\" alt=\"Share\"\/><\/a><script type=\"text\/javascript\">var hupso_services_t=new Array(\"Twitter\",\"Facebook\",\"Google Plus\",\"Linkedin\",\"Digg\",\"Reddit\");var hupso_toolbar_size_t=\"small\";var hupso_counters_lang = \"en_US\";var hupso_url_t=\"\";var hupso_title_t=\"Splunk + Revision Control (Subversion Example)\";<\/script><script type=\"text\/javascript\" src=\"http:\/\/static.hupso.com\/share\/js\/share_toolbar.js\"><\/script><!-- Hupso Share Buttons --><\/div><h1>Why?<\/h1>\n<p>You might be asking &#8220;Why should I use revision control with Splunk \u2013 I&#8217;m not developing code or anything!&#8221; The thing is, with Splunk you <em>are<\/em> developing code, it&#8217;s just that Splunk does a great job of hiding that fact from you! For example, when you add\/update a saved search or dashboard, Splunk is adding\/updating a text file on the server with that information. This means that we can track those changes and (gasp) <em>document<\/em> those changes as we make them!<\/p>\n<p>Here are just a few of the advantages to doing things this way:<\/p>\n<ul>\n<li>Makes it simple to track <em>what<\/em> you did, <em>when<\/em> you did it and <em>why<\/em> you did it.<\/li>\n<li>Instills some <em>discipline<\/em> in your Splunk development.<\/li>\n<li>Gives you the ability (combined with a ticketing system) to associate changes to requests.<\/li>\n<li>Makes it easier to be experimental via features like branching. Want to see if a revamp somewhere works better? Go ahead \u2013 it&#8217;s easy to roll back to a known good configuration <em>while<\/em> retaining all your experiments!<!--more--><\/li>\n<\/ul>\n<h1>Subversion Setup\/Install<\/h1>\n<p>You can skip this section if you already have a working Subversion server. If not, I highly recommend <a href=\"http:\/\/www.visualsvn.com\/server\/download\/\">VisualSVN<\/a> Server (Windows-only, unfortunately) as a dead-simple solution combined with <a href=\"http:\/\/tortoisesvn.net\/downloads.html\">TortoiseSVN<\/a> as a graphical shell. Once you&#8217;ve downloaded it and run setup (the defaults should work just fine for most situations) you&#8217;ll want to do two more things \u2013 create a repository (stick with the default structure):<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Splunk Create new repository\" src=\"http:\/\/www.ossintegrators.com\/blog\/wp-content\/uploads\/2013\/03\/031013_2201_SplunkRevis1.png\" width=\"403\" height=\"215\" \/><\/p>\n<p>\u2026 and add at least one user so that you can track the commits.<\/p>\n<h1>Splunk Directory Overview<\/h1>\n<p>The screenshot below shows the basic Splunk directory structure. As you can see, it&#8217;s very &#8220;Unix-y&#8221;, which actually works in our favor when figuring out what to put in Subversion.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Splunk directory structure\" src=\"http:\/\/www.ossintegrators.com\/blog\/wp-content\/uploads\/2013\/03\/031013_2201_SplunkRevis2.png\" width=\"158\" height=\"141\" \/><\/p>\n<p>In general (and likely the case for most installations) we will end up only putting items in the &#8220;etc&#8221; directory into the repository and use ignore properties to keep unneeded\/unwanted files out.<\/p>\n<h1>Using Subversion with Splunk<\/h1>\n<p>Before you start this section you&#8217;ll need a repository (new or existing one). The first step is to perform a checkout <em>into<\/em> the Splunk program directory. This might seem odd, but this is how you add files to a new repository from an existing directory structure. I recommend adding the &#8220;etc&#8221; directory at the &#8220;top&#8221; such that the repository will look like this:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Splunk Subversion repository structure\" src=\"http:\/\/www.ossintegrators.com\/blog\/wp-content\/uploads\/2013\/03\/031013_2201_SplunkRevis3.png\" width=\"166\" height=\"120\" \/><\/p>\n<p>Doing it this way ensures flexibility: while we&#8217;re not putting anything in the &#8220;var&#8221; or &#8220;share&#8221; directory in the repository now, structuring the repository like I recommend allows us to do so in the future. Below is a screenshot of what the checkout dialog will look like. Be sure that the &#8220;Checkout Directory&#8221; (in yellow) is the top-level Splunk directory!<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Splunk Subversion checkout dialog\" src=\"http:\/\/www.ossintegrators.com\/blog\/wp-content\/uploads\/2013\/03\/031013_2201_SplunkRevis4.png\" width=\"465\" height=\"381\" \/><\/p>\n<p>You&#8217;ll get this scary looking dialog, but you can ignore it since the repository is empty!<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Splunk Subversion overwrite dialog\" src=\"http:\/\/www.ossintegrators.com\/blog\/wp-content\/uploads\/2013\/03\/031013_2201_SplunkRevis5.png\" width=\"388\" height=\"169\" \/><\/p>\n<p>Before we add any files, though, you&#8217;ll want to update the global ignore settings for TortoiseSVN. Once you have a fully populated repository you can use svn:ignore properties, but this works best for new repositories. Add the following:<\/p>\n<div>\n<table style=\"border-collapse: collapse;\" border=\"0\">\n<colgroup>\n<col style=\"width: 150px;\" \/>\n<col style=\"width: 474px;\" \/><\/colgroup>\n<tbody valign=\"top\">\n<tr>\n<td style=\"padding-left: 7px; padding-right: 7px; border: solid 0.5pt;\">Ignore Name<\/td>\n<td style=\"padding-left: 7px; padding-right: 7px; border-top: solid 0.5pt; border-left: none; border-bottom: solid 0.5pt; border-right: solid 0.5pt;\">Description<\/td>\n<\/tr>\n<tr>\n<td style=\"padding-left: 7px; padding-right: 7px; border-top: none; border-left: solid 0.5pt; border-bottom: solid 0.5pt; border-right: solid 0.5pt;\">metadata<\/td>\n<td style=\"padding-left: 7px; padding-right: 7px; border-top: none; border-left: none; border-bottom: solid 0.5pt; border-right: solid 0.5pt;\">These directories are Splunk-generated and change often. Don&#8217;t include these because they will cause a lot of &#8220;noise&#8221; in your repository.<\/td>\n<\/tr>\n<tr>\n<td style=\"padding-left: 7px; padding-right: 7px; border-top: none; border-left: solid 0.5pt; border-bottom: solid 0.5pt; border-right: solid 0.5pt;\">history<\/td>\n<td style=\"padding-left: 7px; padding-right: 7px; border-top: none; border-left: none; border-bottom: solid 0.5pt; border-right: solid 0.5pt;\">As the name implies, these directories store history information such as searches you&#8217;ve done etc.<\/td>\n<\/tr>\n<tr>\n<td style=\"padding-left: 7px; padding-right: 7px; border-top: none; border-left: solid 0.5pt; border-bottom: solid 0.5pt; border-right: solid 0.5pt;\">bin<\/td>\n<td style=\"padding-left: 7px; padding-right: 7px; border-top: none; border-left: none; border-bottom: solid 0.5pt; border-right: solid 0.5pt;\">While these are usually Python files (and thus text), they are system-managed so you probably don&#8217;t want them in the repository.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Splunk Subversion global ignore dialog\" src=\"http:\/\/www.ossintegrators.com\/blog\/wp-content\/uploads\/2013\/03\/031013_2201_SplunkRevis6.png\" width=\"624\" height=\"414\" \/><\/p>\n<p>Now we&#8217;re ready to add the actual files and directories to our repository by right-clicking on the &#8220;etc&#8221; directory and choosing TortoiseSVN-&gt;Add. Since we&#8217;ve set things up correctly already just choose OK. All the files we want to track will be added. Once you commit we&#8217;re ready to actually use revision control to help us manage our Splunk instance!<\/p>\n<h1>Some Common Changes<\/h1>\n<p>If you have not used revision control with Splunk before you may not have noticed exactly <em>what<\/em> happens when you make changes in the GUI \u2013 this is a nice side benefit. Let&#8217;s walk through a few scenarios so you can see what Splunk is doing under-the-covers.<\/p>\n<h2>Logging in for the First Time<\/h2>\n<p>Here&#8217;s a screenshot of what changes after you first login with the &#8220;admin\/changeme&#8221; credentials. Notice that Splunk changed a file but also create some directories? This will be a recurring theme.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Splunk Subversion first login results\" src=\"http:\/\/www.ossintegrators.com\/blog\/wp-content\/uploads\/2013\/03\/031013_2201_SplunkRevis7.png\" width=\"463\" height=\"169\" \/><\/p>\n<p>Let&#8217;s add the untracked files and commit:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Splunk Subversion first commit dialog\" src=\"http:\/\/www.ossintegrators.com\/blog\/wp-content\/uploads\/2013\/03\/031013_2201_SplunkRevis8.png\" width=\"329\" height=\"423\" \/><\/p>\n<h2>Add a Data Input<\/h2>\n<p>Let&#8217;s add a directory of SMTP logs and see what changes! Two interesting files were added in this case:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Splunk Subversion adding data input results\" src=\"http:\/\/www.ossintegrators.com\/blog\/wp-content\/uploads\/2013\/03\/031013_2201_SplunkRevis9.png\" width=\"512\" height=\"268\" \/><\/p>\n<p>The &#8220;inputs.conf&#8221; file was added (in yellow) \u2013 this is the really important one for us to track! Notice also that &#8220;viewstates.conf&#8221; was added because I changed the fields I wanted to see.<\/p>\n<p>Be sure to add these directories and commit.<\/p>\n<h2>Add a Saved Search<\/h2>\n<p>Let&#8217;s add a saved search and see what happens\u2026<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Splunk Subversion adding saved search results\" src=\"http:\/\/www.ossintegrators.com\/blog\/wp-content\/uploads\/2013\/03\/031013_2201_SplunkRevis10.png\" width=\"510\" height=\"238\" \/><\/p>\n<p>We get an additional directory with two .conf files in the <em>application<\/em> we created them in \u2013 that&#8217;s important to keep in mind.<\/p>\n<h2>Add a Dashboard<\/h2>\n<p>Let&#8217;s add a dashboard based on the previous Saved Search. What happens?<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Splunk Subversion adding dashboard results\" src=\"http:\/\/www.ossintegrators.com\/blog\/wp-content\/uploads\/2013\/03\/031013_2201_SplunkRevis11.png\" width=\"500\" height=\"235\" \/><\/p>\n<p>Great! This is actually making sense \u2013 but what if I wanted to make this dashboard available to more than just &#8220;me&#8221; (note the \/users\/admin\/* in the screenshot)? Here&#8217;s what happens when I expand the scope to the &#8220;Search&#8221; application (after a commit \u2013 always commit!):<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"Splunk Subversion moving dashboard results\" src=\"http:\/\/www.ossintegrators.com\/blog\/wp-content\/uploads\/2013\/03\/031013_2201_SplunkRevis12.png\" width=\"568\" height=\"254\" \/><\/p>\n<p>Splunk moved the dashboard definition, which made Subversion note that the previously revision controlled file is now missing (yellow to purple)! There is no way to track this change \u2013 so be sure to make good comments!<\/p>\n<h1>Workflow Considerations<\/h1>\n<p>If all you ever use revision control for is to periodically commit your Splunk configuration changes, that is just dandy. However, the real power of revision control comes when you wrap some &#8220;development discipline&#8221; around your changes. At a minimum, I suggest the following:<\/p>\n<ul>\n<li>Use a ticketing system to track all changes, both ones that you make and those that come from &#8220;outside&#8221;. I highly recommend the free Redmine system (especially the <a href=\"http:\/\/www.turnkeylinux.org\/redmine\">Turnkey Linux<\/a> version because it also includes all major revision control systems already built-in) \u2013 it integrates with Subversion (and most others) so that you can have a full 360-degree view of changes (request-&gt;change-&gt;commit-&gt;document).<\/li>\n<li>Make changes in discreet functionality &#8220;chunks&#8221; (as best you can). This allows your commits to be clearer and more trackable by your ticketing system.<\/li>\n<li>Pay close attention when committing to &#8220;new&#8221; directories that get created when you add things to Splunk.<\/li>\n<li>Applications that you download from Splunk will show up in the &#8220;apps&#8221; directory and \u2013 if you use the default settings \u2013 will also end up in the repository. This is OK, but something to keep in mind because you will then have two revision control methods \u2013 the Splunk Apps store and Subversion.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<div style=\"padding-bottom:20px; padding-top:10px;\" class=\"hupso-share-buttons\"><!-- Hupso Share Buttons - http:\/\/www.hupso.com\/share\/ --><a class=\"hupso_toolbar\" href=\"http:\/\/www.hupso.com\/share\/\"><img src=\"http:\/\/static.hupso.com\/share\/buttons\/share-small.png\" border=\"0\" style=\"padding-top:5px; float:left;\" alt=\"Share\"\/><\/a><script type=\"text\/javascript\">var hupso_services_t=new Array(\"Twitter\",\"Facebook\",\"Google Plus\",\"Linkedin\",\"Digg\",\"Reddit\");var hupso_toolbar_size_t=\"small\";var hupso_counters_lang = \"en_US\";var hupso_url_t=\"\";var hupso_title_t=\"Splunk + Revision Control (Subversion Example)\";<\/script><script type=\"text\/javascript\" src=\"http:\/\/static.hupso.com\/share\/js\/share_toolbar.js\"><\/script><!-- Hupso Share Buttons --><\/div><p>Why? You might be asking &#8220;Why should I use revision control with Splunk \u2013 I&#8217;m not developing code or anything!&#8221; The thing is, with Splunk you are developing code, it&#8217;s just that Splunk does a great job of hiding that fact from you! For example, when you add\/update a saved search or dashboard, Splunk is [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[6],"tags":[27,26,53,25],"class_list":["post-254","post","type-post","status-publish","format-standard","hentry","category-splunk","tag-redmine","tag-revision-control","tag-splunk","tag-subversion"],"_links":{"self":[{"href":"http:\/\/www.ossintegrators.com\/blog\/wp-json\/wp\/v2\/posts\/254","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.ossintegrators.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.ossintegrators.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.ossintegrators.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.ossintegrators.com\/blog\/wp-json\/wp\/v2\/comments?post=254"}],"version-history":[{"count":4,"href":"http:\/\/www.ossintegrators.com\/blog\/wp-json\/wp\/v2\/posts\/254\/revisions"}],"predecessor-version":[{"id":259,"href":"http:\/\/www.ossintegrators.com\/blog\/wp-json\/wp\/v2\/posts\/254\/revisions\/259"}],"wp:attachment":[{"href":"http:\/\/www.ossintegrators.com\/blog\/wp-json\/wp\/v2\/media?parent=254"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.ossintegrators.com\/blog\/wp-json\/wp\/v2\/categories?post=254"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.ossintegrators.com\/blog\/wp-json\/wp\/v2\/tags?post=254"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}