Difference between revisions of "BXadmin:BX Migration stuff"

From CCGB
Jump to: navigation, search
(Created page with "This is to serve as a list of services and responsibilites to be either migrated to ITS or ECoS IT, or decom'd. This is going to be rather long, and many services listed here wi...")
 
 
(8 intermediate revisions by the same user not shown)
Line 2: Line 2:
  
 
This is going to be rather long, and many services listed here will have a long list of requirements.
 
This is going to be rather long, and many services listed here will have a long list of requirements.
 +
 +
= Services =
 +
 +
* AFS
 +
** Statistics:
 +
*** 306 user entries in prdb
 +
*** 98 groups in prdb
 +
*** 669 volumes (not including BK and RO) across 17 fileservers and 42 vice partitions
 +
*** ~100TB total vice partition space dedicated to AFS, most in Solaris+ZFS with at least LZJB compression
 +
*** ~30TB disk space used, after compression
 +
*** PTS users and groups sync'd from LDAP (bxAccount and bxAFSGroup objectClasses, bxAFSGroupId and bxAFSPTSName attr's)
 +
** Stored in afs:
 +
*** Websites (31 vhosts, 8 physical servers), config files, log files, SSL certs, webalizer and associated scripts (not currently enabled, needs to go into cfengine)
 +
*** rancid, cacti, ganglia-web
 +
*** afs-pkg (/afs/bx/pkg and /afs/bx/software)
 +
*** WPKG and Munki xml repos and software packages/installers
 +
*** local YUM repositories, miscellaneous mirrors (ftp.bx.psu.edu/(software|mirrors)
 +
*** Sun Grid Engine with AFS token integration (/afs/bx/service/sge/prod)
 +
*** DNS zone files (config files are in cfengine)
 +
*** cfengine master repo
 +
*** cosign - html files, cgi's, mod_cosign.so
 +
*** Private and public FTP via k5start'd vsftpd
 +
*** Mailman archive files and list config files
 +
*** Rt 3.8.8, built for 64-bit linux and Solaris 10 SPARC (via @sys symlinks)
 +
*** Several 10's of TB of research data (/afs/bx/depot/data/)
 +
*** 214 user home directories, 5.6TB total, average 26GB in size, biggest is 414GB
 +
* Mail
 +
** imap.bx.psu.edu: dovecot IMAP, with GSSAPI auth. Works with BX.PSU.EDU and dce.psu.edu
 +
** Squirrelmail and Horde/IMP for web based access
 +
** Mail routing, aliases, mailer config, and per-user mail forwarding stored in LDAP
 +
** Sendmail on Solaris 10 SPARC for incoming MX's, delivery via procmail+Maildir, sendmail process is run under k5start
 +
** Outgoing via password auth sendmail (smtp.bx.psu.edu)
 +
** MX and smtp.bx are filtered via milter over IPv6 to amavisd running on wash and zoe
 +
** Handles bx.psu.edu directly, and contains bio.cse.psu.edu virtual users and corresponding aliases
 +
** amavisd uses both clamd and spamassassin, and does dkim and DomainKey signing/verification
 +
* Mailing lists
 +
** Mailman (lists.psu.edu). Some of it is in AFS, so it gets k5start'd
 +
** lists.bx.psu.edu accepts mail from the MXes
 +
* LDAP
 +
** OpenLDAP
 +
** dc=bx,dc=psu,dc=edu - syncrepl replicated between 3 machines, single-master. HDB backend
 +
*** Contains mail, users, groups, Apple OD stuff (cn=apple), RADIUS, DHCP, automount maps, sudoers
 +
*** SASL/GSSAPI binds, specifically to allow user-managed groups and mail forwarding edits (mailRoutingAddress) by regular users
 +
*** User RFC2307bis groups
 +
** dc=dce,dc=psu,dc=edu - syncrepl, HDB backend. Contains stub entries for cosign+mod_authnz_ldap functionality with dce users
 +
** dc=psu,dc=edu - fake directory, Perl backend. Returns fake entries for FOPS users
 +
* DNS
 +
** Nothing special. BIND 9 on Solaris SPARC. Many many special entries, SRV records, AAAA records, etc.
 +
* cfengine
 +
** git repo lives in AFS
 +
** cfservd is k5start'd
 +
** Controls almost EVERYTHING
 +
** Needs to be split into separate prod and master branches for proper testing
 +
** Configured to manage Solaris (OpenCSW), OSX (macports), Linux (Centos5, SL6, Debian Squeeze)
 +
* Munki
 +
** Installs pkg's, dmg's, etc on OS X. Currently set up to run on 10.5, 10.6, 10.7
 +
** pkginfo files (xml) edited directly in AFS. Should be moved to git with separate master and prod branches for testing
 +
* WPKG
 +
** Installs all the Windows packages. Targeted at 64-bit Win7 as setup right now.
 +
** Runs via BAT script at startup, configured via AD (BX-WPKG-prod and BX-WPKG-master GPO's)
 +
* IPP
 +
** CUPS running on a (rather old) Fedora VM. All printers are on RFC1918 network. Firewall ACLs allow access to 631/tcp from psu wireless subnet.
 +
** No authentication or page counting, but SPNEGO auth could, in theory, be used
 +
* RT
 +
** Version 3.8.8. Protected behind our cosign
 +
** installed using shipwright, with some minor modifications
 +
* SGE
 +
** Scheduling for most of the clusters
 +
** Lives in AFS, with AFS token integration (set_token.sh and get_token.sh)
 +
* Kerberos
 +
** Heimdal 1.3.3. iprop over ipv6.
 +
** One-way trust between BX.PSU.EDU and dce.psu.edu
 +
** 589 principals
 +
* Cosign
 +
** weblogin.bx.psu.edu
 +
** Supported login realms:
 +
*** fops.psu.edu(R_U = abc123@fops.psu.edu, R_R=fops.psu.edu)
 +
*** dce.psu.edu (R_U = abc123, R_R=dce.psu.edu)
 +
*** BX.PSU.EDU (R_U = abc123, R_R = BX.PSU.EDU)
 +
** cosignd and monster run on 3 different machines (river, jayne, simon)
 +
** Supports SPNEGO via background AJAX mojo that redirects to /negotiate if SPNEGO is deemed to be working. /negotiate is then protected with mod_auth_kerb (see cosign.bx.psu.edu-ssl vhost file)
 +
** Several local patches to disable kdc verification, and change some variables around so we compare certain fields in cosign.conf before RE substitution
 +
* NTP
 +
** ntpd from OpenCSW on Solaris 10 SPARC
 +
* MySQL
 +
** mysql on early.bx.psu.edu has DBs for RADIUS accounting, core websites, RT, etc
 +
** mysql on badger.bx.psu.edu has research related databases
 +
* RADIUS
 +
** Supports switch login and enable passwords on the HPs and Dell PowerConnect (the 10gig switch)
 +
** 802.1X on the wired network, supporting both BX.PSU.EDU and dce.psu.edu realms
 +
** Also does MAC-based authentication
 +
** Assigns ports into vlans based on MAC or username
 +
** Everything is stored in LDAP
 +
* Monitoring
 +
** Syslog - syslog-ng
 +
** RANCID - does switch config diff'ing
 +
** arpwatch
 +
** nagios - 327 services, 91 hosts
 +
** cacti - has the weathermap plugin installed. WM config files need to be updated for the current network layour
 +
** gangila
 +
* FTP
 +
** vsftpd
 +
** k5start'd so it has access to afs (svc/ftp group)
 +
** Does anonymous and private logins for moving research data back and forth for collaborators that don't have full BX accounts or AFS at their site
 +
* DHCP
 +
** LDAP-based host, networks, etc
 +
* conserver
 +
** Provides unified console access to Sun, IBM, Dell, HP, and any standards-based IPMI Serial-over-Lan interface
 +
* Wiki
 +
** MediaWiki with ACL, LDAP_Group, and REMOTE_USER extensions (on my github account)
 +
* afs-backup.pl (/afs/bx/service/afs/backup)
 +
** Reads in volmountsDB (generated every 12 hours on all fileservers using the custom volmounts command courtesy Tom Keiser)
 +
** Generates dsm.sys file with appropriate VirtualMountpoint stanzas for each mountpoint in the cell
 +
** Selects which volumes to backup based on volume or path regex
 +
** Selects policy based on volume or path regex
 +
** Dumps current VLDB state
 +
** Optionally dumps ACLS
 +
** Runs dsmc incremental against selected mountpoints
 +
** Should be updated to use the new volscan utility, currently only in openafs master, probably will be included in OpenAFS 2.0 (?)
 +
* Networking
 +
** HAHAHA. see [[BXadmin:Network]]
 +
= To AIT? =
 +
 +
= To ECoS IT =
 +
 +
<acl>grettad a, gja2 a</acl>

Latest revision as of 15:15, 6 December 2011

This is to serve as a list of services and responsibilites to be either migrated to ITS or ECoS IT, or decom'd.

This is going to be rather long, and many services listed here will have a long list of requirements.

Services

  • AFS
    • Statistics:
      • 306 user entries in prdb
      • 98 groups in prdb
      • 669 volumes (not including BK and RO) across 17 fileservers and 42 vice partitions
      • ~100TB total vice partition space dedicated to AFS, most in Solaris+ZFS with at least LZJB compression
      • ~30TB disk space used, after compression
      • PTS users and groups sync'd from LDAP (bxAccount and bxAFSGroup objectClasses, bxAFSGroupId and bxAFSPTSName attr's)
    • Stored in afs:
      • Websites (31 vhosts, 8 physical servers), config files, log files, SSL certs, webalizer and associated scripts (not currently enabled, needs to go into cfengine)
      • rancid, cacti, ganglia-web
      • afs-pkg (/afs/bx/pkg and /afs/bx/software)
      • WPKG and Munki xml repos and software packages/installers
      • local YUM repositories, miscellaneous mirrors (ftp.bx.psu.edu/(software|mirrors)
      • Sun Grid Engine with AFS token integration (/afs/bx/service/sge/prod)
      • DNS zone files (config files are in cfengine)
      • cfengine master repo
      • cosign - html files, cgi's, mod_cosign.so
      • Private and public FTP via k5start'd vsftpd
      • Mailman archive files and list config files
      • Rt 3.8.8, built for 64-bit linux and Solaris 10 SPARC (via @sys symlinks)
      • Several 10's of TB of research data (/afs/bx/depot/data/)
      • 214 user home directories, 5.6TB total, average 26GB in size, biggest is 414GB
  • Mail
    • imap.bx.psu.edu: dovecot IMAP, with GSSAPI auth. Works with BX.PSU.EDU and dce.psu.edu
    • Squirrelmail and Horde/IMP for web based access
    • Mail routing, aliases, mailer config, and per-user mail forwarding stored in LDAP
    • Sendmail on Solaris 10 SPARC for incoming MX's, delivery via procmail+Maildir, sendmail process is run under k5start
    • Outgoing via password auth sendmail (smtp.bx.psu.edu)
    • MX and smtp.bx are filtered via milter over IPv6 to amavisd running on wash and zoe
    • Handles bx.psu.edu directly, and contains bio.cse.psu.edu virtual users and corresponding aliases
    • amavisd uses both clamd and spamassassin, and does dkim and DomainKey signing/verification
  • Mailing lists
    • Mailman (lists.psu.edu). Some of it is in AFS, so it gets k5start'd
    • lists.bx.psu.edu accepts mail from the MXes
  • LDAP
    • OpenLDAP
    • dc=bx,dc=psu,dc=edu - syncrepl replicated between 3 machines, single-master. HDB backend
      • Contains mail, users, groups, Apple OD stuff (cn=apple), RADIUS, DHCP, automount maps, sudoers
      • SASL/GSSAPI binds, specifically to allow user-managed groups and mail forwarding edits (mailRoutingAddress) by regular users
      • User RFC2307bis groups
    • dc=dce,dc=psu,dc=edu - syncrepl, HDB backend. Contains stub entries for cosign+mod_authnz_ldap functionality with dce users
    • dc=psu,dc=edu - fake directory, Perl backend. Returns fake entries for FOPS users
  • DNS
    • Nothing special. BIND 9 on Solaris SPARC. Many many special entries, SRV records, AAAA records, etc.
  • cfengine
    • git repo lives in AFS
    • cfservd is k5start'd
    • Controls almost EVERYTHING
    • Needs to be split into separate prod and master branches for proper testing
    • Configured to manage Solaris (OpenCSW), OSX (macports), Linux (Centos5, SL6, Debian Squeeze)
  • Munki
    • Installs pkg's, dmg's, etc on OS X. Currently set up to run on 10.5, 10.6, 10.7
    • pkginfo files (xml) edited directly in AFS. Should be moved to git with separate master and prod branches for testing
  • WPKG
    • Installs all the Windows packages. Targeted at 64-bit Win7 as setup right now.
    • Runs via BAT script at startup, configured via AD (BX-WPKG-prod and BX-WPKG-master GPO's)
  • IPP
    • CUPS running on a (rather old) Fedora VM. All printers are on RFC1918 network. Firewall ACLs allow access to 631/tcp from psu wireless subnet.
    • No authentication or page counting, but SPNEGO auth could, in theory, be used
  • RT
    • Version 3.8.8. Protected behind our cosign
    • installed using shipwright, with some minor modifications
  • SGE
    • Scheduling for most of the clusters
    • Lives in AFS, with AFS token integration (set_token.sh and get_token.sh)
  • Kerberos
    • Heimdal 1.3.3. iprop over ipv6.
    • One-way trust between BX.PSU.EDU and dce.psu.edu
    • 589 principals
  • Cosign
    • weblogin.bx.psu.edu
    • Supported login realms:
      • fops.psu.edu(R_U = abc123@fops.psu.edu, R_R=fops.psu.edu)
      • dce.psu.edu (R_U = abc123, R_R=dce.psu.edu)
      • BX.PSU.EDU (R_U = abc123, R_R = BX.PSU.EDU)
    • cosignd and monster run on 3 different machines (river, jayne, simon)
    • Supports SPNEGO via background AJAX mojo that redirects to /negotiate if SPNEGO is deemed to be working. /negotiate is then protected with mod_auth_kerb (see cosign.bx.psu.edu-ssl vhost file)
    • Several local patches to disable kdc verification, and change some variables around so we compare certain fields in cosign.conf before RE substitution
  • NTP
    • ntpd from OpenCSW on Solaris 10 SPARC
  • MySQL
    • mysql on early.bx.psu.edu has DBs for RADIUS accounting, core websites, RT, etc
    • mysql on badger.bx.psu.edu has research related databases
  • RADIUS
    • Supports switch login and enable passwords on the HPs and Dell PowerConnect (the 10gig switch)
    • 802.1X on the wired network, supporting both BX.PSU.EDU and dce.psu.edu realms
    • Also does MAC-based authentication
    • Assigns ports into vlans based on MAC or username
    • Everything is stored in LDAP
  • Monitoring
    • Syslog - syslog-ng
    • RANCID - does switch config diff'ing
    • arpwatch
    • nagios - 327 services, 91 hosts
    • cacti - has the weathermap plugin installed. WM config files need to be updated for the current network layour
    • gangila
  • FTP
    • vsftpd
    • k5start'd so it has access to afs (svc/ftp group)
    • Does anonymous and private logins for moving research data back and forth for collaborators that don't have full BX accounts or AFS at their site
  • DHCP
    • LDAP-based host, networks, etc
  • conserver
    • Provides unified console access to Sun, IBM, Dell, HP, and any standards-based IPMI Serial-over-Lan interface
  • Wiki
    • MediaWiki with ACL, LDAP_Group, and REMOTE_USER extensions (on my github account)
  • afs-backup.pl (/afs/bx/service/afs/backup)
    • Reads in volmountsDB (generated every 12 hours on all fileservers using the custom volmounts command courtesy Tom Keiser)
    • Generates dsm.sys file with appropriate VirtualMountpoint stanzas for each mountpoint in the cell
    • Selects which volumes to backup based on volume or path regex
    • Selects policy based on volume or path regex
    • Dumps current VLDB state
    • Optionally dumps ACLS
    • Runs dsmc incremental against selected mountpoints
    • Should be updated to use the new volscan utility, currently only in openafs master, probably will be included in OpenAFS 2.0 (?)
  • Networking

To AIT?

To ECoS IT

'"`UNIQ--acl-00000000-QINU`"'

Retrieved from "http://wiki.bx.psu.edu/index.php?title=BXadmin:BX_Migration_stuff&oldid=1829"