Difference between revisions of "BXadmin:BX Migration stuff"

From CCGB
Jump to: navigation, search
 
(6 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
This is going to be rather long, and many services listed here will have a long list of requirements.
 
This is going to be rather long, and many services listed here will have a long list of requirements.
  
 +
= Services =
  
 
* AFS
 
* AFS
Line 8: Line 9:
 
*** 306 user entries in prdb
 
*** 306 user entries in prdb
 
*** 98 groups in prdb
 
*** 98 groups in prdb
−
*** 669 volumes across 17 fileservers and 42 vice partitions
+
*** 669 volumes (not including BK and RO) across 17 fileservers and 42 vice partitions
 
*** ~100TB total vice partition space dedicated to AFS, most in Solaris+ZFS with at least LZJB compression
 
*** ~100TB total vice partition space dedicated to AFS, most in Solaris+ZFS with at least LZJB compression
 
*** ~30TB disk space used, after compression
 
*** ~30TB disk space used, after compression
 +
*** PTS users and groups sync'd from LDAP (bxAccount and bxAFSGroup objectClasses, bxAFSGroupId and bxAFSPTSName attr's)
 
** Stored in afs:
 
** Stored in afs:
 
*** Websites (31 vhosts, 8 physical servers), config files, log files, SSL certs, webalizer and associated scripts (not currently enabled, needs to go into cfengine)
 
*** Websites (31 vhosts, 8 physical servers), config files, log files, SSL certs, webalizer and associated scripts (not currently enabled, needs to go into cfengine)
Line 24: Line 26:
 
*** Mailman archive files and list config files
 
*** Mailman archive files and list config files
 
*** Rt 3.8.8, built for 64-bit linux and Solaris 10 SPARC (via @sys symlinks)
 
*** Rt 3.8.8, built for 64-bit linux and Solaris 10 SPARC (via @sys symlinks)
 +
*** Several 10's of TB of research data (/afs/bx/depot/data/)
 +
*** 214 user home directories, 5.6TB total, average 26GB in size, biggest is 414GB
 +
* Mail
 +
** imap.bx.psu.edu: dovecot IMAP, with GSSAPI auth. Works with BX.PSU.EDU and dce.psu.edu
 +
** Squirrelmail and Horde/IMP for web based access
 +
** Mail routing, aliases, mailer config, and per-user mail forwarding stored in LDAP
 +
** Sendmail on Solaris 10 SPARC for incoming MX's, delivery via procmail+Maildir, sendmail process is run under k5start
 +
** Outgoing via password auth sendmail (smtp.bx.psu.edu)
 +
** MX and smtp.bx are filtered via milter over IPv6 to amavisd running on wash and zoe
 +
** Handles bx.psu.edu directly, and contains bio.cse.psu.edu virtual users and corresponding aliases
 +
** amavisd uses both clamd and spamassassin, and does dkim and DomainKey signing/verification
 +
* Mailing lists
 +
** Mailman (lists.psu.edu). Some of it is in AFS, so it gets k5start'd
 +
** lists.bx.psu.edu accepts mail from the MXes
 +
* LDAP
 +
** OpenLDAP
 +
** dc=bx,dc=psu,dc=edu - syncrepl replicated between 3 machines, single-master. HDB backend
 +
*** Contains mail, users, groups, Apple OD stuff (cn=apple), RADIUS, DHCP, automount maps, sudoers
 +
*** SASL/GSSAPI binds, specifically to allow user-managed groups and mail forwarding edits (mailRoutingAddress) by regular users
 +
*** User RFC2307bis groups
 +
** dc=dce,dc=psu,dc=edu - syncrepl, HDB backend. Contains stub entries for cosign+mod_authnz_ldap functionality with dce users
 +
** dc=psu,dc=edu - fake directory, Perl backend. Returns fake entries for FOPS users
 +
* DNS
 +
** Nothing special. BIND 9 on Solaris SPARC. Many many special entries, SRV records, AAAA records, etc.
 +
* cfengine
 +
** git repo lives in AFS
 +
** cfservd is k5start'd
 +
** Controls almost EVERYTHING
 +
** Needs to be split into separate prod and master branches for proper testing
 +
** Configured to manage Solaris (OpenCSW), OSX (macports), Linux (Centos5, SL6, Debian Squeeze)
 +
* Munki
 +
** Installs pkg's, dmg's, etc on OS X. Currently set up to run on 10.5, 10.6, 10.7
 +
** pkginfo files (xml) edited directly in AFS. Should be moved to git with separate master and prod branches for testing
 +
* WPKG
 +
** Installs all the Windows packages. Targeted at 64-bit Win7 as setup right now.
 +
** Runs via BAT script at startup, configured via AD (BX-WPKG-prod and BX-WPKG-master GPO's)
 +
* IPP
 +
** CUPS running on a (rather old) Fedora VM. All printers are on RFC1918 network. Firewall ACLs allow access to 631/tcp from psu wireless subnet.
 +
** No authentication or page counting, but SPNEGO auth could, in theory, be used
 +
* RT
 +
** Version 3.8.8. Protected behind our cosign
 +
** installed using shipwright, with some minor modifications
 +
* SGE
 +
** Scheduling for most of the clusters
 +
** Lives in AFS, with AFS token integration (set_token.sh and get_token.sh)
 +
* Kerberos
 +
** Heimdal 1.3.3. iprop over ipv6.
 +
** One-way trust between BX.PSU.EDU and dce.psu.edu
 +
** 589 principals
 +
* Cosign
 +
** weblogin.bx.psu.edu
 +
** Supported login realms:
 +
*** fops.psu.edu(R_U = abc123@fops.psu.edu, R_R=fops.psu.edu)
 +
*** dce.psu.edu (R_U = abc123, R_R=dce.psu.edu)
 +
*** BX.PSU.EDU (R_U = abc123, R_R = BX.PSU.EDU)
 +
** cosignd and monster run on 3 different machines (river, jayne, simon)
 +
** Supports SPNEGO via background AJAX mojo that redirects to /negotiate if SPNEGO is deemed to be working. /negotiate is then protected with mod_auth_kerb (see cosign.bx.psu.edu-ssl vhost file)
 +
** Several local patches to disable kdc verification, and change some variables around so we compare certain fields in cosign.conf before RE substitution
 +
* NTP
 +
** ntpd from OpenCSW on Solaris 10 SPARC
 +
* MySQL
 +
** mysql on early.bx.psu.edu has DBs for RADIUS accounting, core websites, RT, etc
 +
** mysql on badger.bx.psu.edu has research related databases
 +
* RADIUS
 +
** Supports switch login and enable passwords on the HPs and Dell PowerConnect (the 10gig switch)
 +
** 802.1X on the wired network, supporting both BX.PSU.EDU and dce.psu.edu realms
 +
** Also does MAC-based authentication
 +
** Assigns ports into vlans based on MAC or username
 +
** Everything is stored in LDAP
 +
* Monitoring
 +
** Syslog - syslog-ng
 +
** RANCID - does switch config diff'ing
 +
** arpwatch
 +
** nagios - 327 services, 91 hosts
 +
** cacti - has the weathermap plugin installed. WM config files need to be updated for the current network layour
 +
** gangila
 +
* FTP
 +
** vsftpd
 +
** k5start'd so it has access to afs (svc/ftp group)
 +
** Does anonymous and private logins for moving research data back and forth for collaborators that don't have full BX accounts or AFS at their site
 +
* DHCP
 +
** LDAP-based host, networks, etc
 +
* conserver
 +
** Provides unified console access to Sun, IBM, Dell, HP, and any standards-based IPMI Serial-over-Lan interface
 +
* Wiki
 +
** MediaWiki with ACL, LDAP_Group, and REMOTE_USER extensions (on my github account)
 +
* afs-backup.pl (/afs/bx/service/afs/backup)
 +
** Reads in volmountsDB (generated every 12 hours on all fileservers using the custom volmounts command courtesy Tom Keiser)
 +
** Generates dsm.sys file with appropriate VirtualMountpoint stanzas for each mountpoint in the cell
 +
** Selects which volumes to backup based on volume or path regex
 +
** Selects policy based on volume or path regex
 +
** Dumps current VLDB state
 +
** Optionally dumps ACLS
 +
** Runs dsmc incremental against selected mountpoints
 +
** Should be updated to use the new volscan utility, currently only in openafs master, probably will be included in OpenAFS 2.0 (?)
 +
* Networking
 +
** HAHAHA. see [[BXadmin:Network]]
 +
= To AIT? =
 +
 +
= To ECoS IT =
 +
 +
<acl>grettad a, gja2 a</acl>

Latest revision as of 14:15, 6 December 2011

This is to serve as a list of services and responsibilites to be either migrated to ITS or ECoS IT, or decom'd.

This is going to be rather long, and many services listed here will have a long list of requirements.

Services

  • AFS
    • Statistics:
      • 306 user entries in prdb
      • 98 groups in prdb
      • 669 volumes (not including BK and RO) across 17 fileservers and 42 vice partitions
      • ~100TB total vice partition space dedicated to AFS, most in Solaris+ZFS with at least LZJB compression
      • ~30TB disk space used, after compression
      • PTS users and groups sync'd from LDAP (bxAccount and bxAFSGroup objectClasses, bxAFSGroupId and bxAFSPTSName attr's)
    • Stored in afs:
      • Websites (31 vhosts, 8 physical servers), config files, log files, SSL certs, webalizer and associated scripts (not currently enabled, needs to go into cfengine)
      • rancid, cacti, ganglia-web
      • afs-pkg (/afs/bx/pkg and /afs/bx/software)
      • WPKG and Munki xml repos and software packages/installers
      • local YUM repositories, miscellaneous mirrors (ftp.bx.psu.edu/(software|mirrors)
      • Sun Grid Engine with AFS token integration (/afs/bx/service/sge/prod)
      • DNS zone files (config files are in cfengine)
      • cfengine master repo
      • cosign - html files, cgi's, mod_cosign.so
      • Private and public FTP via k5start'd vsftpd
      • Mailman archive files and list config files
      • Rt 3.8.8, built for 64-bit linux and Solaris 10 SPARC (via @sys symlinks)
      • Several 10's of TB of research data (/afs/bx/depot/data/)
      • 214 user home directories, 5.6TB total, average 26GB in size, biggest is 414GB
  • Mail
    • imap.bx.psu.edu: dovecot IMAP, with GSSAPI auth. Works with BX.PSU.EDU and dce.psu.edu
    • Squirrelmail and Horde/IMP for web based access
    • Mail routing, aliases, mailer config, and per-user mail forwarding stored in LDAP
    • Sendmail on Solaris 10 SPARC for incoming MX's, delivery via procmail+Maildir, sendmail process is run under k5start
    • Outgoing via password auth sendmail (smtp.bx.psu.edu)
    • MX and smtp.bx are filtered via milter over IPv6 to amavisd running on wash and zoe
    • Handles bx.psu.edu directly, and contains bio.cse.psu.edu virtual users and corresponding aliases
    • amavisd uses both clamd and spamassassin, and does dkim and DomainKey signing/verification
  • Mailing lists
    • Mailman (lists.psu.edu). Some of it is in AFS, so it gets k5start'd
    • lists.bx.psu.edu accepts mail from the MXes
  • LDAP
    • OpenLDAP
    • dc=bx,dc=psu,dc=edu - syncrepl replicated between 3 machines, single-master. HDB backend
      • Contains mail, users, groups, Apple OD stuff (cn=apple), RADIUS, DHCP, automount maps, sudoers
      • SASL/GSSAPI binds, specifically to allow user-managed groups and mail forwarding edits (mailRoutingAddress) by regular users
      • User RFC2307bis groups
    • dc=dce,dc=psu,dc=edu - syncrepl, HDB backend. Contains stub entries for cosign+mod_authnz_ldap functionality with dce users
    • dc=psu,dc=edu - fake directory, Perl backend. Returns fake entries for FOPS users
  • DNS
    • Nothing special. BIND 9 on Solaris SPARC. Many many special entries, SRV records, AAAA records, etc.
  • cfengine
    • git repo lives in AFS
    • cfservd is k5start'd
    • Controls almost EVERYTHING
    • Needs to be split into separate prod and master branches for proper testing
    • Configured to manage Solaris (OpenCSW), OSX (macports), Linux (Centos5, SL6, Debian Squeeze)
  • Munki
    • Installs pkg's, dmg's, etc on OS X. Currently set up to run on 10.5, 10.6, 10.7
    • pkginfo files (xml) edited directly in AFS. Should be moved to git with separate master and prod branches for testing
  • WPKG
    • Installs all the Windows packages. Targeted at 64-bit Win7 as setup right now.
    • Runs via BAT script at startup, configured via AD (BX-WPKG-prod and BX-WPKG-master GPO's)
  • IPP
    • CUPS running on a (rather old) Fedora VM. All printers are on RFC1918 network. Firewall ACLs allow access to 631/tcp from psu wireless subnet.
    • No authentication or page counting, but SPNEGO auth could, in theory, be used
  • RT
    • Version 3.8.8. Protected behind our cosign
    • installed using shipwright, with some minor modifications
  • SGE
    • Scheduling for most of the clusters
    • Lives in AFS, with AFS token integration (set_token.sh and get_token.sh)
  • Kerberos
    • Heimdal 1.3.3. iprop over ipv6.
    • One-way trust between BX.PSU.EDU and dce.psu.edu
    • 589 principals
  • Cosign
    • weblogin.bx.psu.edu
    • Supported login realms:
      • fops.psu.edu(R_U = abc123@fops.psu.edu, R_R=fops.psu.edu)
      • dce.psu.edu (R_U = abc123, R_R=dce.psu.edu)
      • BX.PSU.EDU (R_U = abc123, R_R = BX.PSU.EDU)
    • cosignd and monster run on 3 different machines (river, jayne, simon)
    • Supports SPNEGO via background AJAX mojo that redirects to /negotiate if SPNEGO is deemed to be working. /negotiate is then protected with mod_auth_kerb (see cosign.bx.psu.edu-ssl vhost file)
    • Several local patches to disable kdc verification, and change some variables around so we compare certain fields in cosign.conf before RE substitution
  • NTP
    • ntpd from OpenCSW on Solaris 10 SPARC
  • MySQL
    • mysql on early.bx.psu.edu has DBs for RADIUS accounting, core websites, RT, etc
    • mysql on badger.bx.psu.edu has research related databases
  • RADIUS
    • Supports switch login and enable passwords on the HPs and Dell PowerConnect (the 10gig switch)
    • 802.1X on the wired network, supporting both BX.PSU.EDU and dce.psu.edu realms
    • Also does MAC-based authentication
    • Assigns ports into vlans based on MAC or username
    • Everything is stored in LDAP
  • Monitoring
    • Syslog - syslog-ng
    • RANCID - does switch config diff'ing
    • arpwatch
    • nagios - 327 services, 91 hosts
    • cacti - has the weathermap plugin installed. WM config files need to be updated for the current network layour
    • gangila
  • FTP
    • vsftpd
    • k5start'd so it has access to afs (svc/ftp group)
    • Does anonymous and private logins for moving research data back and forth for collaborators that don't have full BX accounts or AFS at their site
  • DHCP
    • LDAP-based host, networks, etc
  • conserver
    • Provides unified console access to Sun, IBM, Dell, HP, and any standards-based IPMI Serial-over-Lan interface
  • Wiki
    • MediaWiki with ACL, LDAP_Group, and REMOTE_USER extensions (on my github account)
  • afs-backup.pl (/afs/bx/service/afs/backup)
    • Reads in volmountsDB (generated every 12 hours on all fileservers using the custom volmounts command courtesy Tom Keiser)
    • Generates dsm.sys file with appropriate VirtualMountpoint stanzas for each mountpoint in the cell
    • Selects which volumes to backup based on volume or path regex
    • Selects policy based on volume or path regex
    • Dumps current VLDB state
    • Optionally dumps ACLS
    • Runs dsmc incremental against selected mountpoints
    • Should be updated to use the new volscan utility, currently only in openafs master, probably will be included in OpenAFS 2.0 (?)
  • Networking

To AIT?

To ECoS IT

'"`UNIQ--acl-00000000-QINU`"'

Retrieved from "https://wiki.bx.psu.edu/index.php?title=BXadmin:BX_Migration_stuff&oldid=1829"