UK Meet the Devs – Cloud Workshop

Back by popular demand! We are holding a UK ‘Meet the Developers’ event to focus on Cloud topics. We are very lucky to have Dean Hildebrand, Master Inventor, Cloud Storage Software from IBM over in the UK to lead this session.

IS IT FOR ME? Slightly different to the past meet the devs format, this is a cloud workshop aimed at looking how Spectrum Scale fits in the world of cloud. Rather than being a series of presentations and discussions led by IBM, this workshop aims to look at how Spectrum Scale can be used in cloud environments. This will include using Spectrum Scale as an infrastructure tool to power private cloud deployments. We will also look at the challenges of accessing data from cloud deployments and discuss ways in which this might be accomplished.

If you are currently deploying OpenStack on Spectrum Scale, or plan to in the near future, then this workshop is for you. Also if you currently have Spectrum Scale and are wondering how you might get that data into cloud-enabled workloads or are currently doing so, then again you should attend. To ensure that the workshop is focused, numbers are limited and we will initially be limiting to 2 people per organisation/project/site.

WHAT WILL BE DISCUSSED? Our topics for the day will include on-premise (private) clouds, on-premise self-service (public) clouds, off-premise clouds (Amazon etc.) as well as covering technologies including OpenStack, Docker, Kubernetes and security requirements around multi-tenancy. We probably don’t have all the answers for these, but we’d like to understand the requirements and hear people’s ideas. Please let us know what you would like to discuss when you register.

Arrival is from 10:00 with discussion kicking off from 10:30. The agenda is open discussion though we do aim to talk over a number of key topics. We hope to have the ever popular (though usually late!) pizza for lunch.

WHEN Thursday 20th October 2016 from 10:00 AM to 3:30 PM
WHERE IT Services, University of Birmingham – Elms Road Edgbaston, Birmingham, B15 2TT
REGISTER Please register for the event in advance: https://www.eventbrite.com/e/ssug-meet-the-devs-cloud-workshop-tickets-27725390389 Numbers are limited and we will initially be limiting to 2 people per organisation/project/site.

Spectrum Scale User Group (Australia) Meeting

Venue: Pullman Hotel, Albert Park, Melbourne (in conjunction with eResearch Australia Conference)

Date: Friday 14th October 2016.

To register your place please visit: https://www.eventbrite.com/e/spectrumscale-user-group-australia-gpfsugaus-tickets-26814516942?ref=enivtefor001&invite=MTA1MDg1MTkvY2hyaXMuc2NobGlwYWxpdXNAaXZlYy5vcmcvMA%3D%3D&utm

Thanks to DDN for supporting this event. IBM representatives will be at the meeting to cover various topics, and open dialogue between IBM and customers will be encouraged. As is the culture of the group, no marketing or sales pitches!

Agenda

8:30am – 4:30pm
– Introduction by Dr Neil Stringfellow, Executive Director, Pawsey SuperComputing Centre (Perth). – Day overview, OHS matters etc

  • Speakers: IBM – SpectrumScale Roadmap

Spectrum Scale Users Group Argonne National Lab – June 2016

MidWest Users Group Event

Many thanks to Argonne National Lab (ANL) for hosting our recent Spectrum Scale Users Group Event. It’s nice to have an event in the MidWest given the geography spread in the US. Bob Oesterlin kicked things off with a social event Thursday night so some of us could share stories prior to the actual UG day. Friday morning was focused on IBM presentations and the majority of the afternoon went to user presentations.

To start, we received updates on both Spectrum Scale and ESS from Scott McFadden. Some notable priorities for 2016 include making sure that US customers have the opportunity and channel to give feedback to development in the early phase of the process and shape the result. Scott noted that there’s been great feedback in this respect from the UK users, so you heard it, let your voice be heard US users! There should be some follow up traffic on the mailing lists about that —watch that space if you are interested. There will also be news about more open betas that are accessible in a downloadable VM.

Additionally, input from the PMR and field teams is being leveraged more effectively. IBM is big and there’s recognition that in the past PMR and field information may not have been getting back to the development team effectively.

Security is another focus and an internal security audit is ongoing as mentioned at SC. Ease of use in configuring key management with ISKLM is coming in 4.2.1

During the Problem Determination session many comments were put on screen about how tricky it is to know if a GPFS cluster is healthy, and problem determination is tricky when it’s not. To that end, an mmhealth command is being developed to report on all the key components in a cluster. This should answer the questions of what components to monitor, what command to use to do so and how to interpret the results. The tool takes into consideration all of the interdepencies to report a high-level healty, degraded or unhealthy. mmhealth is being reviewed with user input as it is being developed.

For the GUI tools there were both screen shots and a live demo. The question was asked to “Raise your hand if you monitor waiters?” Lots of hands shot up. The follow up question was “Keep your hand up if fully understand them.” I think Sven was the only one who keep his hand up. The GUI tool is building up a long waiters componetnt to categorize an document waiters.

Sven Oehme gave us an overview of some features coming with 4.2.1. On Scale 4.2 there are > 700 parameters for tuning —many of them undocumented, but still used in production, so in many cases customers have a lot to figure out. It’s difficult for IBM to come up with default settings when the range of hardware and networking capabilties for any given site varies wildly. Still, to make it easier on the customer some auto-tuning capabilities are being added. For example, there will be a new worker thread setting that will auto-tune about 20 other related settings. Care is being taken to make sure that those who want to retain manual settings can do so, and there will be information about this in the documentation. Long term there is a goal to let the admin describe the system and let that inform parameter choices automatically.

The user presentations were interesting and included campus Active Directory Integration, Using GPFS on ZFS, GPFS-HPSS-Integration (GHI) and using AFM as a Burst Buffer. All presentations are available online, check them out.

All presentations are available here: http://www.spectrumscale.org/presentations/

Q&A:

Some questions from the audience included, there were many more, people weren’t shy about interjecting questions:

Q: While there is appreciation for quick development changes, there is a concern for the quality of the releases.
A: This is an area that’s being actively reviewed for improvement and better regression testing to make sure changes don’t negatively impact performance.

Q: How much is compression being used in the wild?
A: Not much production use, but people are interested for future implementation. Generally it takes a year for new features to be adopted in deployment.

Q: With mmbackup, how much data can you backup?
A: It depends largely upon your infrastructure and how much you can parallelize. Multiple TSM servers can be used for Spectrum Scale now, but a discussion of your architecture would be required to answer with a numeric value.

Q: In the monitoring tool, can detailed tracking be seen?
A: Yes, at the granularity of individual filesystem calls, setattr, mkdir, vget, getxattr, etc.

Q: What is the retention of the data that is behind the monitoring tools?
A: It’s configurable, the default is something like 1s resolution for 24 hours, and then it starts getting aggregated and resolution is reduced.

The next in person event in the US is still being planned. Stay tuned.

Spectrum Scale Day @ SPXXL (New York) – May 26th, 2016

Spectrum Scale Day @ SPXXL at New York Academy of Medicine
https://www.spxxl.org/?q=New-York-City-2016

AGENDA

8:30- 9:00 Registration
9:00- 9:45 Top 10 reasons for upgrading from GPFS 3.5 to Spectrum Scale 4.2.1
9:45-11:00 Panel Discussion: My favorite tool for monitoring/reporting
11:00-11:30 -Break-
11:30-12:30 Problem Determination
12:30-13:30 -Lunch-
13:30-14:00 Site report – AWE
14:00-14:30 Spectrum Scale in Life Sciences
14:30-15:00 HPSS Update
15:00-15:30 -Break-
15:30-16:30 Best practices for GPFS or Spectrum Scale Tuning
16:30-17:15 Enhancements for CORAL

UK May 2016 group report

Well, the 2016 UK May user group has finished. It has been a lot of work getting here, and a lot of organisation to get our first 2-day event up and running. Jez commented at the start of his talk on how far the user group has come – six years ago it was 12 people in a room with a bad Skype link, and now we over 100 attendees, nearly 30 speakers and two packed days of user talks, IBM talks and sponsor talks. We even had a VP or two along this year to hear what everyone had to say. We were amazed how fast we “sold out” (we’re free to attend!)!
IBM South Bank
First, a note of thanks, to Claire O’Toole, our group Secretary (and OCFer!) and Ulf Troppens, our IBM development liaison for getting the event together. And also to all the speakers for coming along, talking to us, listening to us and allowing us to challenge them for the future. We brought together speakers and users from across the globe – three continents (and that wasn’t just the IBM speakers!).

We had attendees from across the whole “Spectrum” of sectors, research institutions, VARs, finance and media, and hopefully the attendees were happy with the programme we put together. It was also great to see some of the L1 support guys from the UK – a great opportunity to put a face to a name. This year was the first time we’ve put together a two-day programme and hopefully people learned new things, found it useful and maybe most importantly, that we’ve inspired some people to look at new features of Spectrum Scale, try things out and upgrade from 3.5!

This year we returned to the IBM Client Centre at IBM South Bank, chatting to some of the speakers, it was the first time they’d been to IBM South Bank, and for some their first visit to London – I hope we impressed them with our event!

A bunch of us met up for an informal dinner and drinks the evening before, which was a great way of meeting people, and checking that the IBM speakers flying in had all made it … Claire and I didn’t have to make any contingency plans for the agenda which was nice!

This year we filmed many of the talks, and we’re hoping to post videos from the talks over the next few weeks – we need to cut them down and plan to fix up the slides, in the meantime, many of the slide decks are already posted on the presentations page.

Doris OpeningDay one started with myself and Claire opening the meeting and handing over to Doris Conti (Development Director for Spectrum Scale) to give us our keynote talk. Doris mentioned that she wants the people at the user group to each year to say “Wow” about what they have been doing over the past year, there’s certainly some areas where I think that is the case.

Ulf's Opening talk
Ulf followed on bringing us an overview of the development priorities for 2016 and was followed by Markus giving us a sneak peak of the next GUI release coming this summer. Just before lunch, we finished with the first of our Sponsor talks – Ray from Seagate talked on Spectrum Scale and IoT, some interesting deployments using Spectrum Scale for Windows as an ingest target for CCTV systems.

After lunch, we were met with another sponsor talk. Jez from ArcaStream talked on their Python API and gave a short demo of it in use. Our first user deployment talk, Dave from MilkVFX gave a fascinating overview of their deployment of AFM between their London and Cardiff offices (and their showreel video got a round of applause as well). Next up, a sponsor talk, Mo from NetApp spoke on the E series systems and how they can be deployed for HPC file-systems.

Our second user deployment talk came from Pam at NCAR, talking on their massive deployment changes happening this summer. The final pre-break talk came from IBM with a deep dive look at the Hadoop integration of Spectrum Scale and how the new transparency layer works. Check-out the slides for details on accessing your POSIX file-system data from your Hadoop workloads!

The final set of talks began with a sponsor talk, with Darren from Mellanox giving an overview on how fabric cards could be used in the future to support storage systems. Another two deep dive technical sessions followed, with Robert talking about the Transparent Cloud Tiering product coming later this year, and then Mathias spoke on Problem Determination, the new features in the 4.2.1 release and the new mmhealth command.

We wrapped up the day on time and then many of us headed down to Namco Funscape in County Hall for the evening social event. The bumper cars and bowling proved a massive hit along with the pool and table tennis tables. Ulf and I even checked out the rollercoaster simulator!

Namco Funscape

Day two started a little earlier – we figured most people would already be in London, and it was great to see pretty much everyone make it for the 9:30 start, even with the evening social ….

We kicked off the morning with parallel break-out sessions, with a well-attended talk on licensing from David, and an equally well attended sponsor talk from Laurence at OCF talking on Scale out NAS. I was surprised at how few people (OK only me!) have faced the PVU to Socket conversion … the take-away is to start the process early!

Breakout sessions

Our breakouts continued with Indulis talking on metadata (with real actual numbers and how it works), a session to chat with the GUI developers, and Nils leading a series of Spectrum Protect (TSM!) talks – covering Spectrum Protect Space Management (ILM), Archive (LTFS-EE)SOBAR and using Spectrum Scale as a store for Spectrum Protect. Our other breakout speakers included Madhav on AFM and Olaf and Yohann talking on encryption and compressions. There was also a packed session on monitoring, with Doris opening to the floor for open discussion on what we’d like to see and how to integrate.

Lunch saw the IBM Client Centre people reset the rooms back into a single space for our afternoon slots. Doug asked for a few minutes to present the results of day 1’s survey – how much storage we looked after in the room, and how many people do it … We found amazingly that its 2.9PB/person!

Vic from DDN brought us another sponsor talk, covering adventures in AFM, and looked at their Imperial College London deployment and the massive effect link latency can have on AFM throughput. Gaurang followed up talking on OpenStack integration, covering the Cinder driver and also the object on file work and gave a demo of using object on file with the Manilla driver to have file-based access to a file whilst simultaneously having access to the same file via Swift. The session was wrapped up with Madhav speaking on IBM Life Sciences and how Spectrum Scale can be used to underpin the demanding research needs of the life sciences areas.

Time for another break, followed by Michael from Lenovo, a sponsor talk, clarifying a few post Lenovo divesture myths and talking about development of the GSS solution and how they are still working closely with IBM to deliver the product. We closed the talks with Sven talking about IBM research giving us an insight into what is being thought about for the future and also on some major performance developments which have been driven by major contracts that have a benefit to all.

Doris provided us with some closing remarks and thoughts on the day before Claire and myself wrapped up the day. For me, it has been a great but tiring few days. We’ve had a lot of positive feedback so it was well worth it!

Oh, and we event started discussions about finding a bigger venue for 2017. We’re still hoping to bring together a meet the devs later this year and also hope to coordinate another (much!) smaller user group meeting towards the end of the year.

Finally, a note of thanks to the sponsors for the event, without their support we couldn’t have made the evening event happen.

ArcaStream
DDN
IBM
Lenovo
Mellanox
NetApp
OCF
Seagate