Agenda item
Update report to the Joint IT Committee of the London Boroughs of Brent, Lewisham and Southwark
This report provides an update on the performance of 3-council ICT Shared Service established in November 2017.
Minutes:
Prod Sarigianis (Head of Digital Services, Brent Council) introduced the report to the Joint Committee updating Members on key performance areas in relation to the ICT Shared Service.
As a starting point, Prod Sarigianis provided an update on the levels of service disruption which had recently been experienced, with the reasons identified as follows:
· The need to undertake a security review of the network which had resulted in a number of essential changes being made;
· The need to deliver, within a shorter than usual timescale the replacement of all telephony services for Brent;
· A series of outages caused by problems with the Virgin Media Business (VMB) link connecting the shared services datacentres.
Members noted that as a result of the above issues performance in relation to the volume of open calls and call resolution had significantly suffered which combined with the recent major outages affecting Brent & Lewisham had, it was recognised, raised concerns regarding the service. A series of measure were now in the process of being implemented, designed to stabilise and improve the service moving forward.
In terms of detailed performance updates, Members noted the summary of key performance indicators for the service across all three Councils up to May 2018, which had been circulated with the update report. The performance update had confirmed the increase in call volumes within Brent and Lewisham anticipated as part of the telephony rollout and also reflected the increase in number of open calls caused as a result of the various factors linked to the major service disruption already reported, which were currently in the process of being cleared. This process had also created an impact in terms of reduced levels of call resolution within the required timescales. Whilst noting the update provided Members were also advised that performance for June (not covered within the report) had shown significant levels of improvement with it anticipated that the backlog of open calls would have been addressed as part of the next monitor.
In terms of measures now being taken to address the concerns raised, Prod Sarigianis highlighted a number of service related actions on which further detail had been provided within section 3.10 of the report. These had been based around a review of key processes and risk assessments designed to identify lessons learnt and mitigate against future reoccurrence.
The following comments and issues were highlighted by Members in relation to the service actions identified:
(a) An update was sought on progress in addressing the outstanding issues experienced in relation to replacement of telephony services at Brent and Lewisham. Prod Sarigianis advised that the main issue remained the reporting function available to the Customer Contact Centre and on which work was now being finalised to introduce a Performance Dashboard.
(b) Further details were sought on the proposals being developed with the storage expert engaged from the network vendor around optimising systems and setting up more proactive performance alerts. Prod Sarigianis advised that this would initially be based around development of an early warning system to alert staff of any slow-down or performance issues around the thin client system. Whilst criteria had already been established for identifying significant issues, the difficulty had been in maintaining sufficient staffing support to monitor and support the core network on a 24/7 basis. Alongside the work with the network vendor, options were also being explored to procure a more comprehensive and robust level of support for the core network on a 24/7 basis recognising the concerns expressed and way in which demand for services and use of the network had now expanded.
(c) Members supported the escalation of concerns with senior managers at both Virgin and LGfL regarding the connectivity issues being experienced between the shared service datacentres and noted the escalation with BT for the installation of a second interconnect. Pending full resolution of the connectivity issues, Prod Sarigianis assured Members of the interim measures in place which included running all live services from the Brent data centre and development of a number of improvements to the resilience of the data centres including an overall disaster recovery plan.
(d) In terms of the options being explored around additional proactive monitoring for services, Members advised they were keen to ensure measures were in place enabling earlier detection of issues affecting systems (particularly the thin client) rather than relying on reactive logging of calls by service users. Prod Sarigianis advised options being explored included the use of system and storage monitoring software as well as a move away from the thin client solution.
Members were then provided with an update on progress with Phase 2 of the Southwark Transition (data centre migration), which Prod Sarigianis advised had now needed to be moved back to September due to issues with upgrades of the links to the existing data centre provided by LGfL/Virgin. Whilst noting that the original deadline for completion of this phase of the transition had been October, Members were keen to ensure the September deadline was achievable. Prod Sarigianis advised that at this stage plans remained on track with Peter Gadsdon (Director of Policy, Performance & Partnerships – Brent Council) highlighting the alternative link also being sourced though BT in order to provide additional resilience supported by the longer term aim to move to a shared data centre. Whilst limited by the number of providers available, Members were assured of the robust challenge provided to LGfL/Virgin in relation to the level of service currently being received given its impact on the shared service.
In terms of updates on other projects, Members noted (as detailed in the report):
· The progress made on the shared service restructure;
· The planned rollout of laptops in Brent was due to commence in July, with the process feeding development of the projects for Southwark and Lewisham;
· Planning for the telephony rollout in Southwark had now commenced with implementation to be undertaken only once the review of lessons learnt from the project in Brent & Lewisham had been reviewed and all remediation actions had been completed;
· The progress in rollout of Wi-Fi provision across remaining sites within Southwark;
Prod Sarigianis concluded the update with a brief outline of progress on various procurement activity related to the shared service, with Members noting:
· As it had not been possible to agree on the selection of a preferred bidder, the procurement for Regulatory Services Software would now be pursued as a separate process by each Authority;
· Completion of the reverse auction for end user devices (desktops and laptops) for all three Councils with the final contract awarded for a range of Lenovo devices through XMA;
· The progress being made in preparation for the three Council tender of office and bulk printing, for which contract award was anticipated in November;
· The progress in seeking to procure a single contract for the supply of Children’s & Educational Services Software within Brent and Lewisham, which was expected to be completed by the end of July 18.
The discussion was then opened up in relation to the update and shared service performance summary, with the following issues raised:
(a) Members queried the reasons for performance against Priority 2 incidents appearing to be lower than Priority 1 and 3 under the current monitor. In response Prod Sarigianis felt it was important to recognise the relatively low number of Priority 2 incidents and their nature, which often related to performance issues affecting a number of users and required more detailed investigation to identify the underlying issue. Whilst the majority of these incidents were resolved within the required timescale those where it was not possible to meet the target would have a bigger impact on the performance targets than the higher volume and more routine type incidents. In response to a further query, Members were advised that the use of severity was not currently included as part of the assessment of an incident as these were already subject to detailed criteria and definition.
(b) In relation to the update provided around the restructure of the shared service, further details were sought as to where the vacancies remaining to be filled were currently located. Prod Sarigianis confirmed that whilst the largest staff base was at Brent local pressures had been experienced across all three boroughs with staff being rotated to assist.
(c) Members sought details on the sampling of calls undertaken and monitoring of the types of incidents being logged as a means of identifying specific trends or issues. Prod Sarigianis advised this was being currently explored as part of the move towards more proactive monitoring for all services. In terms of the way calls logged were assessed, he advised this would involve an initial assessment and triage process in order to ensure incidents were being assigned on the basis of the correct priority.
(d) In response to a specific query relating to Phase 2 of the Southwark Transition, Prod Sarigianis advised that problems were not anticipated in terms of the capacity of the datacentre following completion of the migration process. Details were also sought on the projects currently on hold affecting Lewisham as a result of the actions being taken to stabilise the system operating environment as a result of the recent major incidents. Prod Sarigianis advised that the measures in place involved a tighter regime for the approval of new projects based on assessment of their priority and impact with, as an example, network upgrade work currently on hold where it was not felt to be critical in terms of impact on end users.
(e) In terms of the planned rollout of laptops, Members were keen to explore the potential for this to contribute towards the business continuity planning process within each borough.. In response Prod Sarigianis outlined the benefits alongside the rollout of the Microsoft Office 365 cloud based solution from a business continuity perspective in relation to the creation of less dependency on access to the main IT infrastructure and more flexible patterns of working. In terms of timescales he confirmed that the rollout of laptops in Brent was shortly due to commence alongside pilots in Lewisham and Southwark with migration to Microsoft Office 365 being planned across all three boroughs for the end of 2018. The use of laptops had already been successfully rolled out for the LGA.
(f) In relation to the rollout of Wi-Fi across all three boroughs, concerns were highlighted regarding intermittent performance issues currently being experienced within Lewisham Town Hall. Prod Sarigianis advised he was already aware of the issue with a fix in place to address.
(g) As a final issue, details were sought on the measures in place to tackle concerns relating to cyber security. Whilst recognising the concerns raised, given the rapidly changing nature of threats, Prod Sarigianis was able to assure Members on the measures in place to maintain security of the network and systems managed through the shared service. This involved a programme of regular testing and external verification along with the provision of additional staffing resource as part of the current restructure to focus on system security. In addition Peter Gadsdon highlighted the work being undertaken within Brent to develop a cyber security strategy along with a review of the findings from a recent LGA cyber security audit, which would be shared across the three boroughs.
As no further issues were raised the Chair thanked Prod Sarigianis for the update provided and it was RESOLVED:
(1) To note the updates provided and actions being taken in relation to the ongoing performance and delivery of the shared service, as detailed within section 3 of the report; and
(2) To note the contents and updates provided in relation to the Performance Pack, circulated as Appendix 1 of the report.
Supporting documents:
- Joint Committee of Brent, Lewisham & Southwark - Update Report, item 6. PDF 195 KB
- Appendix 1 - ICT Performance Pack June 2018, item 6. PDF 4 MB