So, a few days ago Brian Krebs posted an article on his blog called “Are Your Google Groups Leaking Data?“. This article reached me while I was chilling in the sun but it did not really suprise me as I was researching on the very same topic right before going on my vacation. Actually I wanted to release this information a few weeks ago but mutually agreed with the guys from Google to wait for it until we had a chance for a direct talk which was scheduled right after my vacation (a.k.a today).
At the current point in time there is not much left to be added as you can find most of the interesting information already published in the article from the researchers at Kenna Security and in Brian’s write-up (or you can have a look in the original advisory from Redlock which is even dating back to July 2017). However as I already had my stuff written up I wanted to get it out as well. At the very least it may help to spread out the word and make sure that affected Google customers remediate the issue.
For me it all started a few weeks ago with a colleague pointing me out to a post by @JGamblin on Twitter:
I hate to keep harping on this but you really need to make sure you are not leaking sensitive data through Google Groups if your organization uses them. https://t.co/QdsdefaGAd
— Jerry Gamblin (@JGamblin) May 8, 2018
As I am currently working with different organizations which are using the G Suite on a corporate level I wanted to have a closer look at the matter and was shocked with the results. As luck would have it we had a representative from Google Cloud’s Professional Service at our office so that I could share my results with him directly. He was quite concerned with the matter and decided to hook me up with the PO for Google Groups directly. As I was about to hit my vacation I promised to deliver a quick write-up concerning my obervations and we scheduled a phone call and a disclosure for when I would return.
As I was sitting on a big pile of juicy loot I decided to contact some of the affected organizations manually but quickly realized that it was too much effort for me to contact all of them. As I was confident that the information would get into more and more hands I wanted to bring the topic up to Google before disclosing the issue further. At the very least the articles mentioned above proofed me right on this…
So, today I finally had the chance to talk with Google and offered them my point of view on the matter. Overall it was a quite constructive chat and they assured me that they have taken actions in order to directly contact and inform their customers who might be affected by the issue. Right now I can confirm that only a small subset of the affected customers have fixed this.
During our talk Google agreed on my point of view that the overall presentation of privacy implications within the web-frontend is not very clean. Given that, they also confirmed that right now they are evaluating howto improve the visual representation. So expect a blinking 90’s gif, marquee text and an alarm sound soon!
We also discussed some other ways they had in mind of how the overall security / privacy controlls of the solution could be enhanced. As most of the stuff would require larger changes on the overall architecture, I personally feel it is unlikely they are going into this direction any time soon.
Well, all that is left for me to do now, is to once again appeal to G Suite administrators to check their privacy configurations and make sure to have a look at Google’s recommendations, as well as to pay attention to the article published by Kenna Security. They did a good job in describing the overall problem, the potential impact as well as the mitigation process. So I will not pickup those topics here. If you want to get in touch with me, feel free to hit me up on Twitter @0xB455
TL;DR
If you are using Google Groups and your domain is configured to “Public to the internet” you should audit the visibility of each Group. Even though you believe that your sensitive groups are properly protected (as they might not be listed in your public directory index), they still might be publicly accessible without any authentication. Currently multiple thousand domains are affected…
So without further ado: here is the little write-up from May which I supplied to Google (featuring fancy lolcat output – simply because it rulez):
=== BOF ===
Get the party started
Hi folks, as described earlier today the original root cause for my investigation was the fact that we came across an issue where it was possible to publicly access confidential content within Google Groups without being authenticated within our domain. We raised the issue with our internal G Suite admins and they jumped on having a look at the privacy settings for our domain.
After sprinkling some magic dust they replied back to me that they applied proper changes towards the privacy configurations within our domain.
So I had another look and noted that our admins applied changes towards the visibility of groups within the index directory for our Google Groups domain. Seemed good to me as it was no longer readable without a proper authentication. This was accomplished by revoking the listing option from each group.
However I found that it was still possible to publicly access confidential content by either having direct links towards the individual groups or by simply utilizing the search functionality.
I immediately replied back to our admins and we had a more detailed look at the privacy settings again. After some digging through the permission settings we finally were able to determine that some users accidentally configured the permissions for their groups by selecting the option “All organization members”.
As we were running our domain with the “Public on the Internet” permission, this ultimately resulted in the fact that unauthenticated users are capable to access the misconfigured groups:
Also consider a scenario where users within a private Google Groups domain are getting used to apply the “all organization members” option to their groups. Later on in time the “Outside this domain – access to groups” configuration gets switched to “Public on the Internet” without anyone noticing that individual groups are now being exposed towards the public.
Eventually we ended up with a very bad feeling in our stomach and I asked our admins to switch the domain to complete private mode. We will now reconsider our use-cases related to the G Suite products and perform proper risk assessments for them.
Seems shady
Okay let’s have a detailed breakdown what went wrong here.
- we are visiting a publicly accessible domain within Google Groups
- we are validating which groups are listed within the domain
- we are trying to access groups with juicy names and descriptions; no luck… sadface!
- we search for sensitive content within the group; no luck again… sadface!
- we search for sensitive content within the whole domain; et voilà!
data anonymized for public disclosure
- we visit the unlisted group and have a look around …
data anonymized for public disclosure
- … and get our hands on some nice information … profit!
data anonymized for public disclosure
What could possibly go wrong? ¯\_(ツ)_/¯
As I was pretty sure that we were not the only people on the planet running into those unwanted configurations I decided to perform an evaluation of the Alexa top 1 million popular websites. Therefor a colleague and I quickly hacked up a crawler with some python code and within several hours the tool was able to identify about ~6.000 valid domains of Google G Suite customers which were generally running their Google Groups in public mode:
Next up I decided to have a closer look and fired up some sample queries within the individual domains in order to identify Google customers who unintentionally misconfigured their privacy settings in a similar manner as we did it before. Basically I was utilizing some search patterns in order to identify unlisted groups and later on validated which of those were not explicitly included within the overall index (therefore likely to be of a confidential nature).
I was still thinking about the implication of my findings and decided to do some data mining within those groups, trying to evaluate the possibilities of actually gaining access to confidential and sensitive content. Soon after performing some very basic reconnaissance I was sitting on all kinds of sensitive data of Google customers… PayPal accounts which had larger balances waiting to be withdrawn, private keys and certificates used for trusted communication, credentials for web-services or cloud/network infrastructure, confidential employee information, all sorts of financial information and last but not least a lot of GDPR relevant personal data from end customers and private persons…
From the ~6.000 domains nearly 30% (~1800 domains) were actually leaking sensitive data. As a PoC here is an extract showing some of the loot which I was able to identify, feel free to check up the links for yourself:
data anonymized for public disclosure
Please consider that we are barely talking about around 6.000 potential targets that I identified by using a common domain list. The real footprint of affected domains and customers would be somewhat higher then what is included within the Alexa toplist. A malicious and determined threat actor aiming for exploiting the issue in a larger scale will likely hit more domains then the ones I hastily evaluated with the crawler. I am confident that we can extrapolate this number into at least a few thousand additional domains.
Room for improvement
So here is what I recommend in order to mitigate the issue.
First of all: let your affected customers know that they might have a problem with their data being publicly exposed.
As a second: I believe that Google should improve the user interface for their customers and get rid of misleading or intransparent aspects within the UI. Whenever someone is about to set a privacy setting which could lead to complete public exposure there should be a proper warning presented. Basically: whenever users are in danger of exposing content from their domain towards the public internet Google should turn on the red lights and an alarm signal…
Currently the assets for managing the G Suite on a larger scale are limited. We found that by tinkering with tools such as Google Apps Manager (GAM) it is possible to run a batched evaluation of all objects within the G Suite domain. But implementing some kind of native audit feature within the webfrontend would be a huge benefit for administrators.
Conclusion
Depending on the individual scenario users and administrators of Google Groups might unknowingly run into misconfiguration of their privacy settings. This can result in data leaks. For companies whose organization and communication heavily rely on using Google Groups this can have a huge impact.
Digging through all the relevant privacy settings within the G Suite seems not very intuitive and can be a challenging task for admins. Cleaning up or auditing an existing and eventually misconfigured domain might be a challenging task as well. Making use of tools like the Google Apps Manager can mitigate the hassle.
I understand that the discussed issue is not relying within the core functionality of Google Groups per se and that this is no technical security issue within the product. It rather is related towards configuration errors which are unintentionally introduced by users. Handling the entangled collection of privacy controls and assessing their implications might be challenging.
I personally feel that it is Google’s duty to make their customers (admins and users) more aware of potential risks and implications regarding the individual privacy controls within the G Suite. I feel that Google needs to consider how the ramifications of those controls can be presented towards the user in a better and more transparent way.