Every organisation has vulnerabilities in its digital infrastructure, including in its Domain Name System (DNS). In the Protecting Public Sector Domains team in the Central Digital and Data Office (CDDO) we work to identify and fix those vulnerabilities before our adversaries find them. We’d also like to do that for other kinds of vulnerabilities, related to email and web services. This isn’t easy, but it’s a goal we’re working towards.
Our monitoring tools find misconfiguration and vulnerability data from a variety of services, and we’re gradually expanding our scope and capability. Once we have the data, we have found the hardest part is getting the information into the hands of the person who can fix it.
To that end we are running a Domain Data Sharing programme to send the vulnerability data we collect directly to public sector organisations via their SIEM (Security Information and Event Management) or other systems.
Last year, we ran a discovery programme to look for a way to share all our vulnerability data, not just the biggest and most urgent problems. We talked to people in public sector organisations who manage and fix domain issues, or operate vulnerability management or other teams that work every day to fix these kinds of problems. We found that:
We also found that SIEM adoption is growing. These are systems that collect and analyse data from different sources like network devices or servers, and external feeds like ours, and use them to spot security issues. These are toolsets that can handle the volume of data we offer in a way that’s useful to our users.
In light of our learnings during the discovery phase, we've got a Domain Data Sharing beta programme running right now that will:
We’re also working with NCSC to include our data in ACD services in the future.
So if you have struggled with the kind of problems we found in our discovery, or you'd just like to get a free feed of domain, web, and email vulnerabilities for your public sector organisation, we’d like you to join our Domain Data Sharing beta programme. Get in touch with us at support@domains.gov.uk to sign up.
In an unprecedented age where secure and efficient collaboration is paramount, the Central Digital and Data Office has taken a significant stride forward by collaborating with Microsoft and the National Cyber Security Centre (NCSC) to develop comprehensive guidance for the UK government organisations that are using Microsoft 365.
The guidance has been designed by digital, data and technology professionals across government to empower and support each other in alignment with the Mission Four: Efficient, Secure and Sustainable Technology of the 2022 to 2025 roadmap: Transforming for a digital future.
This blog outlines three pieces of guidance available that are key to enhancing secure configuration, external collaboration and information protection within the Microsoft 365 environment.
The Microsoft 365 Guidance for UK Government: Secure Configuration Alignment guides government departments on how to securely configure their Microsoft 365 Platform in alignment with the latest NCSC advice in the journey towards fortified cybersecurity.
By adhering to these guidelines, digital, data and technology professionals can understand how the features and capabilities in Microsoft 365 can be used to ensure that a common bar has been achieved for their tenant and promote a secure digital ecosystem.
It is no surprise that collaboration lies at the heart of modern government operations. The Microsoft 365 Guidance for UK Government: External Collaboration is a huge step forwards for colleagues that rely on effective collaboration and rapid communication with teams across HM government.
This guidance provides a baseline configuration for the Microsoft 365 platform enabling key features such as real-time file sharing and co-authoring across government departments, efficient meeting scheduling through free-busy calendar sharing, consistent instant messaging (IM), video conferencing and the creation of cross-governmental Microsoft Teams sites.
This blueprint not only streamlines communication, but also paves the way for more efficient and impactful cross-department collaboration.
In an era of data-driven governance, safeguarding sensitive information is non-negotiable.
The Microsoft 365 Guidance for UK Government: Information Protection sets forth a standard configuration for sensitivity labels and data loss prevention. This guidance caters to organisations aiming to implement the OFFICIAL tier of the updated Government Security Classification Policy (GSCP) using Microsoft Purview, a part of the Microsoft 365 suite.
By adhering to this guidance, government departments can confidently navigate the intricacies of data protection while harnessing the full potential of their information assets.
For digital, data and technology colleagues navigating the implementation of our Microsoft 365 guidance within the UK Government, assistance and insights are just an email away. Connect with the dedicated OneIT team at oneit@digital.cabinet-office.gov.uk for tailored support and clarifications.
For those curious about the integration of these guidelines within your department, your IT Department is perfectly placed to provide further details. Reach out to them to learn about the plans and strategies for adopting the Microsoft 365 guidance, tailored to the UK government.
Stay informed and engaged as your department takes strides towards secure and efficient collaboration.
Technology has revolutionised every aspect of our society and our economy, including the way that we deliver our public services, helping to make people’s lives easier and safer. Security vulnerabilities are discovered all the time online and people want to be able to report them directly to the organisation responsible. That’s why we are advocating for the use of security.txt as a standardised way of doing just that. One of the most important elements of vulnerability disclosure, and a challenge for the finder, is understanding who to contact.
Security.txt describes a text file that advertises the organisation’s vulnerability disclosure process so that someone can quickly find all of the information needed to report a vulnerability. It is a voluntary standard for internet users set by the Internet Engineering Task Force (RFC 9116).
Security.txt will serve the government in its aim to become more resilient in its online security by making it easier for anyone to report vulnerabilities they have found. Quick, easy and secure reporting directly to the affected department speeds up the triage and remediation time and reduces the risk of compromise, such as reporting of a vulnerable web server so it can be remediated before being exploited. The security.txt was endorsed by the Data Standards Authority in March 2023.
Benefits to government departments & finders
The ability to receive, respond and ultimately fix a reported vulnerability is essential to providing secure products and services. Being open to receiving vulnerability reports helps departments engage constructively with those who find them - ‘finders’. Engaging with finders can be a source of valuable information that would otherwise be missed or require additional time and effort to discover.
Vulnerability disclosure policy
Departments should define what they expect from someone reporting a vulnerability, as well as what they will do in response, by providing a clear policy. This enables the department and the finder to confidently work within an agreed framework.
In its basic form, a vulnerability disclosure policy should contain the following information:
How to implement security.txt
Security.txt is a plaintext file that should be published in the “/.well-known” directory of the domain root.
The file contains three key fields:
CONTACT: How finders should report vulnerabilities. For example, email or secure web form.
POLICY: A link to the department’s vulnerability disclosure policy.
EXPIRES: Indicates the date and time after which the data contained in the "security.txt" file is considered stale and should not be used. The value of this field is formatted according to the Internet profile of [ISO.8601] as defined in [RFC3339]. It is recommended that the value of this field be less than a year into the future to avoid staleness.
The ENCRYPTION field is optional and should link to the PGP public key you wish to be used for encrypted communication.
The National Cyber Security Centre (NCSC) has published the NCSC Vulnerability Disclosure Toolkit that provides information on how to implement security.txt as well as an example vulnerability disclosure policy.
We posted previously about our work removing gsi-family domains from the public sector and why we are doing it.
This work is now complete and over 3,500 domains have been removed including:
We will continue to monitor the remaining domains for issues, but this work has removed the bulk of the risk of email spoofing and domain hijacking from misconfiguration for these domains. We expect that further work may be required in the future to completely remove gsi.gov.uk and to look at the domains that remain within the PSN (Public Services Network).
Since our previous blog, we’ve made the following changes:
Please note that no PSN-facing zones were changed during this work.
There are 83 .gsi.gov.uk domains remaining. If you still have one of these domains you should take steps now to remove it. If it still works for email, we recommend you change settings to start rejecting inbound email. You can also choose to include a bounce-back message giving senders the correct address.
You should also check public facing websites or documentation for mentions of gsi-family domains and remove them.
If you have any questions around this work or need advice on removing a domain get in touch with us at support@domains.gov.uk.
If you have a domain that has been suspended or removed as part of this work and you need it restored contact Nominet directly on psnsupport@nominet.uk or 01865 332493.
Last updated: 29 March 2023
Most gsi-family domain names (gsi.gov.uk, gse.gov.uk, gcsx.gov.uk or gsx.gov.uk) are scheduled for removal from their internet-facing zones by the beginning of April.
A core pillar of the Transforming for a Digital Future strategy is delivering efficient, secure and sustainable technology, and, at CDDO’s Securing Government Services team we're working hard to clean up and remove legacy services.
Some public sector organisations have previously used .gsi.gov.uk, x.gsi.gov.uk, .gsx.gov.uk, .gse.gov.uk and .gcsx.gov.uk to email each other in a secure way. However, the current email standards and guidance mean they can now get better security sending the same email over the internet rather than using the Public Services Network (PSN).
The PSN, where these gsi-family domains were used, is in the process of being wound down, and we officially stopped using these domains in 2019. The PSN email relay they depended on meanwhile was shut down in 2021.
People are reluctant to remove old domain names, often because they are concerned there might be a forgotten service that depends on the domain. This means these old domains can get neglected and become vulnerable to spoofing and malicious attacks.
Many gsi-family domains still exist in both internet and PSN-facing zones. Most are dormant, some are misconfigured, and all are targeted heavily for email spoofing. As a result we plan to remove most of the internet-facing zones entirely at the beginning of April.
As a starting point we’ve added more protection to reduce the impact, in the form of DMARC records to protect the apex domains and prevent the spoofing of domains that don’t exist. DMARC records tell the receiving email service what the legitimate senders are for that domain. If an email comes from somewhere else it gets marked as spam.
This blog previously stated we would suspend and remove PSN-facing zones in addition to the internet-facing zones. This is no longer the case, although we will review the option to do this in the future.
Most of the domains appear to be dead already, pointing to services that do not exist or reject queries. It is possible there are still some dependencies we don’t know about. Email may be being routed through to modern systems to provide continuity for old addresses.
If you still have one of these domains and it still works for email, start rejecting inbound email. You can also choose to include a bounce-back message giving senders the correct address. It will be removed at the beginning of April so it would be good to give anyone still using it some notice.
You should also check public facing websites or documentation for mentions of gsi-family domains and remove them.
We have identified a small number of domains that are operating internet facing services that can't yet migrate to a new domain. We have excluded these domains from the suspension and removal process.
If you have a domain you think you will need beyond the beginning of April, get in touch with us now at support@domains.gov.uk so we can work out a solution.
If you have a domain that has been suspended or removed as part of this work and you need it restored contact Nominet directly on psnsupport@nominet.uk or 01865 332493.
]]>The Future Networks for Government (FN4G) team, part of the Cabinet Office’s Central Digital and Data Office (CDDO), is helping organisations to migrate away from the Public Services Network (PSN) as it’s increasingly hard to secure. The team is encouraging organisations to migrate to modern network solutions, which offer more competitive commercial terms as well as greater flexibility and scalability.
To help us get a better picture of what our users and suppliers need, we’re running a pre-market engagement and industry consultation. We’re partnering with Innopsis, which will run the pre-market engagement and coordinate the consultation with industry.
We’ll discuss, test, and iterate technical models, service models, and different ways of working. We hope the outcome will mean PSN users can improve their network security posture and implement a more modern network and telecommunications services environment.
More specifically, we want the pre-market engagement process to:
We’ll be looking into setting up work groups jointly chaired by the Cabinet Office and Innopsis. Initially focusing on 4 areas - technical, transition and commercial, security, and service management - the work groups will make sure the pre-market engagement meets its deadlines.
The work groups will also be responsible for:
We’ll also be setting up a steering committee to coordinate activities across the work groups, and take overall responsibility for taking these proposals forward to the Cabinet Office for final approval.
We’ll be hosting a pre-market engagement launch event where you can learn more about FN4G, what it will mean for future public sector networking and telecommunications procurements, and how you can shape it.
The Securing Government Services team at the Central Digital and Data Office recently encountered, and fixed, a small bug with the way some government domains handle their Sender Policy Framework (SPF) records.
SPF allows a domain owner to specify which email services are authorised to send email on their behalf. For example, the website example.gov.uk might use Outlook for email - so they can create a TXT record in their DNS which says:
<span style="font-weight: 400">"v=spf1 include:spf.protection.outlook.com -all"</span>
This tells the internet that if someone tries to send an email purporting to come from example.gov.uk, it will only be valid if it has been sent from the Outlook server. If it comes from a different server, it must be rejected.
This prevents people from sending spoofed emails which impersonate legitimate government services.
SPF records need to be written in a precise format. If the syntax is even slightly different from the specification, the record is invalid and spoofed emails will be able to be delivered.
Our team recently noticed a record which was syntactically correct - but was, somehow, still being marked as invalid. This was a serious risk and could have let a malicious actor send emails as though they were from a specific government service.
The SPF record looked like this:
<span style="font-weight: 400">"v=spf1 include:example.com include:spfprotectionoutlook.com -all"</span>
The SPF checker works on multiple levels. The first is a simple conformance check - that is, seeing if the record is written in the correct syntax. This record was correct.
The second level is to evaluate the included domains. This involves doing a DNS lookup on the domain. And this is where the problem was.
The second domain was misspelt. When a lookup was performed on that domain, it returned an error - NXDOMAIN.
Because this recursive check failed, it resulted in what the specification calls a "permerror" - which means the domain's published records could not be correctly interpreted.
Even though the record has the correct syntax, and even though other included domains validated correctly, because a single domain didn't exist the entire SPF record failed validation.
The specification is complicated and we had initially expected that a partially matching set of includes would still validate and the SPF would be valid. A thorough reading of the specification shows that is not the case.
The issue was fixed by correcting the spelling of the incorrect domain. We do not believe this error was actively exploited by malicious parties.
We learned several lessons from this incident:
For more information, you can read our guidance on how to set up government email services securely.
]]>We constantly monitor the GOV.UK analytics browser stats and every 6 months we update the ‘Designing for different browsers and devices’ page in the Service Manual. This page gives service teams an idea of the minimum browser support their service should support.
The last Internet Explorer change to the Service Manual was in June 2018, when we changed our testing requirements for Internet Explorer 8, 9 and 10. The reasons these changes occurred is due to our rule that we only support the browsers the top 95% of people use. This is a data-driven process so it's as inclusive as possible to users of GOV.UK. To quote from the Service Manual:
This list in the table is based on usage statistics for GOV.UK and represents approximately 95% of the most popular browsers. It’s updated in January and June every year.
As of June 2022, Internet Explorer 11 (IE11) dropped below the monthly 95% threshold that's required to keep it on the Service Manual minimum browser support table. This also happens to coincide with the announcement from Microsoft that IE11 was retired on 15 June 2022. Beyond this point, Microsoft will be gradually migrating IE11 users on Windows 10 to their newer browser, Edge.
In April 2022 GOV.UK had 390,000 visitors who used IE11 - 0.65% of the total visitors. We expect this to drop dramatically after the 15 June because a vast majority of IE11 visitors were using Windows 10 - so will be automatically upgraded to Edge.
IE11 was first released in October 2013, so it is coming up to 9 years old. It isn't considered a 'modern browser' (a browser that automatically updates approximately every 4-6 weeks) as its updates are tied to the operating system updates (apart from iOS, iPadOS and Macintosh Safari that are currently the exception to this rule). This makes IE11 significantly less secure than other browsers like Microsoft Edge and Google Chrome. With Microsoft migrating IE11 users to their Edge, they will improve both security and overall web performance for these remaining users.
If you are a service team, does this mean you can stop supporting IE11 users from now? No, it doesn't. This is a data-driven decision, and the Service Manual simply lists the minimum browser support that a government service should cater for.
So for example, if your service has user data (such as analytics, user research) to show that you still have a large percentage of IE11 users, then you should continue to support and test in IE11 for these users. The use of the Progressive Enhancement methodology can help with this task. This support need is likely to be for a short period of time, given Microsoft's strategy to migrate IE11 users to Edge.
Yes we can. The GOV.UK Design System team currently supports Internet Explorer all the way down to version 8. Their plans for the future are to fork GOV.UK Frontend into 2 major versions:
This allows service teams to continue to support whatever version of IE their users use. Over time all service teams should plan to migrate to version 5.x.x as it will receive all the latest components and features.
Version 5.x.x will have the advantage of improved web performance and less ongoing maintenance for service teams.
Subscribe to Technology in Government and Inside GOV.UK to keep up to date with our work
]]>In April the GOV.UK Pay infrastructure team noticed some of our test payments were failing intermittently. Payments made by real users in the production environment were unaffected, but the test errors occurred often enough to start blocking our deployment pipeline. The issue lay in our Connector app’s ‘connection time to live’ settings. Here’s how we fixed it.
GOV.UK Pay makes it easy for public sector organisations to take payments by providing an API that connects to various third-party payment providers such as Worldpay and Stripe.
We have different levels of testing for GOV.UK Pay. Mostly we simulate (or ‘stub out’) calls to third-party APIs. To test critical user journeys, like making a payment, we carry out ‘smoke tests’ where we make calls to the Worldpay and Stripe test environments to check our integration is working correctly.
When our smoke tests to Worldpay started failing, with around 1 in 12 test payments giving a ‘connection reset’ error, we raised the issue with Worldpay support. Worldpay said they’d added a firewall in front of their test environment API. They recommended we check that our requests were not relying on calling a fixed IP rather than using DNS resolution. We ruled this out as the problem. So what was happening?
Worldpay were planning to roll out the new firewall change to their live environment at the end of June, so we needed to find a solution to prevent the error impacting real users' payments.
While we investigated we added a basic retry mechanism for the intermittent smoke test failures so GOV.UK Pay developers weren’t blocked from their normal processes. We also made sure our internal stakeholders were aware of the issue and the potential impact on paying users.
We focused on 2 areas of GOV.UK Pay’s infrastructure that were likely to be the cause. They were the:
We ruled out the egress proxy being the cause of the problems by temporarily removing it from our test environment and having Connector make requests directly to the Worldpay API.
We felt this was safe enough, as our test environment contains no real card data. The only transactions which pass through this environment use specially designated test card numbers. All the data which would be considered personal information is fake data and no real user information can enter this environment.
This meant some delicate changes to the existing AWS networking we had set up in the test environment, as previously we had wanted to ensure that Connector definitely could not make direct requests to the outside world.
Unfortunately, even without the egress proxy in place, the errors continued. So we focused our investigation on Connector.
First we increased the HTTP log level on the Connector app to DEBUG, on our test environment only. Normally we avoid logging this level of information, in case we inadvertently log personal data, API keys or card details.
Again, we agreed this would be acceptable as long as it was limited to the test environment, and the code which defines the infrastructure would only allow these changes to be applied in the testing environment. This meant we could get a level of fine detail about the requests Connector was making to Worldpay, including the headers of the requests/responses, and which connection threads were being used.
When we compared the detailed DEBUG-level logs for a successful smoke test payment with those of a failed smoke test payment, we could see that on the failed payments Connector was not even getting as far as sending its POST request to the Worldpay API. The connection was closed and shut down before this could happen. This told us it was likely to be an issue with stale HTTP connections in the pool.
The Connector app uses the Dropwizard/Jersey client, a wrapper around the Apache HTTP client, which is configurable with a number of connection settings. To improve performance, the client maintains a pool of HTTP connections it can reuse. The settings determine how many connections can be used at once, how long they should be kept open for, and so on.
At first it looked like the settings were as they should be, however interestingly the KeepAlive
setting was set to zero, meaning connections should have been closed immediately after use, which didn’t match the behaviour we were seeing.
We then found the HttpConnectionManager
class had a connTTL
(‘connection time to live’) setting, which was hardcoded to -1, meaning it would keep connections alive indefinitely. This felt like it explained the behaviour we were seeing. We cross-checked the documentation for Worldpay’s firewall provider, and found it supported connections lasting up to 500 seconds.
The logs showed that for failed smoke test payments, the same connection had been used around 5-6 minutes earlier on a previous, successful smoke test. It was reasonable to expect the Worldpay firewall to treat this connection as stale, and refuse the request. This theory was backed up by the fact that when smoke tests were run in quick succession (1 minute apart rather than 5 minutes), they tended to pass without errors.
We set the connTTL
setting to 60 seconds, well under the firewall limit, but long enough that it should not affect Connector’s performance. We deployed the change to the test environment, and ran our set of smoke tests. They passed! More importantly, they passed consistently and the connection reset errors no longer appeared.
Once we deployed the fix we made sure we reverted our earlier changes to the DEBUG log level, and removed the temporary networking that allowed us to bypass the egress proxy to safeguard personal data. We kept our smoke test retries in the deployment pipeline to help unblock developers suffering from flaky tests in future. Finally, we fed back our findings to Worldpay to help other users who might be experiencing the same problem.
]]>In December 2021, the Data Standards Authority (DSA), part of the Central Digital and Data Office (CDDO), published a case study about how NHS Digital used API (Application Programming Interface) management to support APIs at scale.
The case study focuses on a project by NHS Digital’s API team, which aimed to “make integration easier” for developers looking to connect to NHS Digital services. It explores how the team used an API management platform to centralise authorisation, documentation and support. The team also took the opportunity to modernise and improve some of these specific areas as well. For example, they published documentation online for the first time to make it easier to access and manage. Additionally, the team rolled out 24-hour support, since many of the APIs on the platform support critical healthcare services.
The case study also outlines the team’s work to redevelop the Personal Demographics Service (PDS) API as an exemplar to showcase the management platform’s capabilities. They built the new PDS API as a RESTful API, and also introduced other modernisations such as serving the API over the open internet and using the Fast Healthcare Interoperability Resources (FHIR) standard for healthcare technology.
The DSA, which is responsible for central government API guidance, produced this case study as part of our goal to promote the use of APIs across the public sector.
Throughout the process, we worked closely with NHS Digital by:
If you’re a product owner, technical lead or another kind of technologist working on a team that builds and runs APIs, this case study will help you:
Read the full case study and share your thoughts and feedback with us by emailing the DSA at data-standards-authority@digital.cabinet-office.gov.uk.
If your team or organisation has a project, initiative or program that could help others in the public sector, a case study could be a great way to showcase your learnings and outcomes for teams to learn from.
To get an idea of what makes a good case study, check out the technology case studies. You can also contact the Case Studies team by emailing technical-writers@digital.cabinet-office.gov.uk or join the #technology-case-studies channel in the cross-government Slack community.
]]>