FileShareDFSProject20160608 < CF

CF Web>InternalProjects>SecureWeb-basedFile-sharingSystem-backingStorage>FileShareDFSProject20160608 (2016-06-06, LoriPaniak) (raw view)

EditAttach

Meeting: 2016-06-09 13:00

Attended: ldpaniak (project manager), a2brenna, cscflab, rblander, jjohnson

Objectives: 
   * Decide on general architecture of service connections to DFS and external clients.
   * Decide on details of DFS configuration, software and hardware.
   

Already under work:
   * high-speed dedicated storage network (Devon/Dan/Lori)

To be determined: 
   * Expected typical/max load of FileShare/OwnCloud gateway in production - how much hardware do we need for active/passive configuration? Containers OK?

Discussion:
   * Review of motivation for SCS OwnCloud/DFS with Math reps.
   * Motivation to give options to replace NetApp in the future seen as valid. 
   * Math has functional, restricted usage OwnCloud system in operation. Looking to expand it in the near future.
   * It appears OwnCloud has option of internal encryption of data so only encrypted bits get to backing storage (DFS).
   * Warning on problems with disparate data formats on backing storage (UTF-8, C.UTF-8...).  Leaving these details to dedicated service providers independent of DFS seen as a positive.
   * Importance of real-time, actionable load monitoring, notifications, predictive failure warnings stressed.
   * General importance of systems in warranty during production agreed.
   * M3 and MC buildings are on same campus power grid.  Questions for HA of DFS and services.
   
Decisions:
   * All end-user services will be mediated by systems/containers/VMs attached to the DFS 40GbE ring network.  No direct end-user access to DFS products will be provided.
   * Mediating services decouples service upgrades, maintenance, security and configuration from DFS core functionality.  This modularity will be invaluable as the number and type of services using DFS products increases.
   * Mediating services allows for per-service level of data encryption and isolation.

  
Next meeting: [[FileShareDFSProject20160600][2016-08-1300]]
Deliverables:
cscflab (Nathan Fish): Investigate options for multi-server (parallel) NFS system that utilizes DFS products (block, glusterfs, cephFS, etc) to provide Kerberized NFS to Math/CS client systems.

a2brenna: Build latest Ceph system on 16.04 with three nodes.  Demonstrate cluster configuration at command line including features (eg. add/remove OSD etc.)

ldpaniak: Build gluster cluster for demo of addition/remove of bricks(~OSD). Use gluster 3.8(?)


jjohnson recommends attendance of Hari Chotara (hchotara@uwaterloo.ca) who has been working on DFS options for Math.



-- Main.LoriPaniak - 2016-06-06

Topic revision: r1 - 2016-06-06 - LoriPaniak

Information in this area is meant for use by CSCF staff and is not official documentation, but anybody who is interested is welcome to use it if they find it useful.

Other Webs

My links
- People
- CERAS
- WatForm
- Tetherless lab
- Ubuntu Main.HowTo
- eDocs
- RGG NE notes
- RGG
- CS infrastructure
- Grad images

Edit