Meeting 3 February 2016, 2pm

Attended: drallen, dlgawley a2brenna fhgunn ldpaniak

Agenda:

  • Timeline for project
    • Next round of milestones
  • Items to be assigned
  • Brief summary since the last meeting

Timeline for project

  • Daniel worked up a timing estimate; we've reviewed it as a group with one minor addition. That schedule puts us:
    • moving first database (inventory) Thursday Feb. 11 (soft date)
    • moving the rest of databases Monday Feb. 22 (soft date)
    • wrapping up Thursday March 3rd. (soft date)

New Milestones

  • Benchmark current mysql.cs performance - initial low-level tests by Lori: reset the due-date to tomorrow.
    • MUST be certain that low-level testing won't cause undue problems to marmoset or faculty-recruiting (Daniel to look at faculty-recruiting schedule; Lori to ask Nick re: Marmoset).
  • Application-level benchmarking of current mysql.cs: to be done by Daniel, Friday 5 Feb.
  • Testing HA failover: Anthony and Fraser, Friday 5 Feb.

Items to be assigned:

  • Testing failover of saltified database (Anthony and Fraser).
  • Establish monitoring of the HA service: resource, latency, health (Anthony will research, can start as soon as saltification is done)

Brief summary since the last meeting:

  • Daniel confirmed with Dave that the server room high-speed network interconnects were racked and ready for firmware updating. Lori has done that step; they now have passwords and updated firmware.
  • Fraser had a casual conversation with Ken, who agreed that asynch replication was much better than what we have, and a reasonable goal for the project.
    • Management has had a formal discussion with Ken. He is now on the same page as us; and is pleased that we're doing this with project management.
  • Lori, Dave, Clayton, and Daniel discussed the "synch vs. asynch" question; Lori did parallel investigation into synchronous clustering.
    • Lori did a full round of testing the weekend of Jan 30th, documented in ST#85594, to indicate synchronous would be possible- but only with a fairly different hardware setup than we've considered, and therefore out of scope for this project this term.
      • "Mysql Cluster" won't work effectively with three data nodes- which would provide no redundancy. It would require four or more; requiring fast network and lots of storage.
      • It would also require a management server (low requirements of CPU/storage/networking)
      • and two or more SQL nodes (high requirements for CPU/networking, low requirements of storage).
  • Milestones: by today:
    • mysql configuration by salt - Anthony: documented in ST#85594 Feb 3 2016 14:04
      • access via root from linux.cscf to mysql-10{2,4,6}.cs
    • Benchmark current mysql.cs performance - initial low-level tests by Lori: reset the due-date to tomorrow.
      • MUST be certain that low-level testing won't cause undue problems to marmoset or faculty-recruiting (Daniel to look at faculty-recruiting schedule; Lori to ask Nick re: Marmoset).

  • Agreed that for now, we will back up both master and slave. This might change later, but the cost is low for additional assurance.
  • Agreed to add an item to saltification: adding a few lines for a shorewall firewall. We should block all but the expected mysql traffic (and ssh from linux.cscf).
  • Agreed that monitoring MIGHT extend to using percona; Antony will saltify percona configuration, as well as shorewall.
  • Noting that tuning mysql parameters will include choosing mount options - to be done by Dave and Anthony.

  • Noting that phase two, the student cluster, could possibly be set to authenticate using active directory rather than a standalone auth setup? However, we might decide to put marmoset on the second cluster, and set up the rest of student databases on the non-HA hardware currently running mysql.cs.

-- DanielAllen - 2016-02-01

Topic revision: r6 - 2016-02-17 - DanielAllen
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback