Hiya! About a days I wrote about small personal programs, and I talked about that
it is going to even be fun to make expend of “secret” undocumented APIs the place you wish to replica your
cookies out of the browser to gain access to them.

About a folks asked how one can conclude this, so I needed to show veil how because
it’s gorgeous straightforward. We’ll also recount a small bit about what can stagger
injurious, ethical factors, and the scheme this applies to your undocumented APIs.

Let’s say, let’s expend Google Hangouts. I’m picking this now not because it’s the
most priceless example (I deem there’s an pleasant API which will seemingly be great extra
vivid to make expend of), but because many web sites the place right here’s truly priceless are
smaller web sites that are extra susceptible to abuse. So we’re apt going to make expend of
Google Hangouts because I’m 100% obvious that the Google Hangouts backend is
designed to be resilient to this style of poking round.

Let’s gain began!

step 1: survey in developer instruments for a promising JSON response

I birth out by going to https://hangouts.google.com, opening the community tab in
Firefox developer instruments and wanting out for JSON responses. That you just can expend Chrome developer instruments too.

Here’s what that looks delight in

The ask is a decent candidate if it says “json” within the “Kind” column”

I needed to survey round for a whereas until I stumbled on one thing attention-grabbing, but
within the kill I stumbled on a “folks” endpoint that looks to approach files about
my contacts. Sounds fun, let’s keep a peek at that.

step 2: replica as cURL

Subsequent, I apt click on the ask I’m drawn to, and click on “Copy” -> “Copy as cURL”.

Then I paste the curl expose in my terminal and escape it. Here’s what happens.

$ curl 'https://folks-pa.clients6.google.com/v2/folks/?key=REDACTED' -X POST ........ (a bunch of headers removed)
Warning: Binary output can mess up your terminal. Employ "--output -" to tell 
Warning: curl to output it to your terminal anyway, or help in thoughts "--output 
Warning: " to keep to a file.

You would be pondering – that’s queer, what’s this “binary output can mess up
your terminal” error? That’s because by default, browsers ship an
Bring collectively-Encoding: gzip, deflate header to the server, to gain compressed
output.

Lets decompress it by piping the output to gunzip, but I pick up it extra vivid
to apt now not ship that header. So let’s do away with some irrelevant headers.

Here’s the fleshy curl expose line that I got from the browser. There’s lots right here!
I birth out by splitting up the ask with backslashes () in insist that every header is on a assorted line to originate it less difficult to work with:

curl 'https://folks-pa.clients6.google.com/v2/folks/?key=REDACTED' 
-X POST 
-H 'Shopper-Agent: Mozilla/5.0 (X11; Linux x86_64; rv: 96.0) Gecko/20100101 Firefox/96.0' 
-H 'Bring collectively: */*' 
-H 'Bring collectively-Language: en' 
-H 'Bring collectively-Encoding: gzip, deflate' 
-H 'X-HTTP-Contrivance-Override: GET' 
-H 'Authorization: SAPISIDHASH REDACTED' 
-H 'Cookie: REDACTED'
-H 'Divulge material-Kind: application/x-www-develop-urlencoded' 
-H 'X-Goog-AuthUser: 0' 
-H 'Foundation: https://hangouts.google.com' 
-H 'Connection: help-alive' 
-H 'Referer: https://hangouts.google.com/' 
-H 'Sec-Get-Dest: empty' 
-H 'Sec-Get-Mode: cors' 
-H 'Sec-Get-Blueprint: same-hassle' 
-H 'Sec-GPC: 1' 
-H 'DNT: 1' 
-H 'Pragma: no-cache' 
-H 'Cache-Regulate: no-cache' 
-H 'TE: trailers' 
--files-raw 'personId=101777723309&personId=1175339043204&personId=1115266537043&personId=116731406166&extensionSet.extensionNames=HANGOUTS_ADDITIONAL_DATA&extensionSet.extensionNames=HANGOUTS_OFF_NETWORK_GAIA_GET&extensionSet.extensionNames=HANGOUTS_PHONE_DATA&includedProfileStates=ADMIN_BLOCKED&includedProfileStates=DELETED&includedProfileStates=PRIVATE_PROFILE&mergedPersonSourceOptions.includeAffinity=CHAT_AUTOCOMPLETE&coreIdParams.useRealtimeNotificationExpandedAcls=apt&requestMask.includeField.paths=person.email&requestMask.includeField.paths=person.gender&requestMask.includeField.paths=person.in_app_reachability&requestMask.includeField.paths=person.metadata&requestMask.includeField.paths=person.title&requestMask.includeField.paths=person.phone&requestMask.includeField.paths=person.portray&requestMask.includeField.paths=person.read_only_profile_info&requestMask.includeField.paths=person.group&requestMask.includeField.paths=person.hassle&requestMask.includeField.paths=person.cover_photo&requestMask.includeContainer=PROFILE&requestMask.includeContainer=DOMAIN_PROFILE&requestMask.includeContainer=CONTACT&key=REDACTED'

This could occasionally appear delight in an overwhelming quantity of stuff on the starting up, but you don’t want
to deem about what any of it manner at this stage. You apt must delete
irrelevant strains.

I in general apt desire out which headers I will delete with trial and blunder – I
help eradicating headers until the ask starts failing. In fundamental you doubtlessly
don’t want Bring collectively*, Referer, Sec-*, DNT, Shopper-Agent, and caching
headers even though.

In this case, I became ready to diminish the ask down to this:

curl 'https://folks-pa.clients6.google.com/v2/folks/?key=REDACTED' 
-X POST 
-H 'Authorization: SAPISIDHASH REDACTED' 
-H 'Divulge material-Kind: application/x-www-develop-urlencoded' 
-H 'Foundation: https://hangouts.google.com' 
-H 'Cookie: REDACTED'
--files-raw 'personId=101777723309&personId=1175339043204&personId=1115266537043&personId=116731406166&extensionSet.extensionNames=HANGOUTS_ADDITIONAL_DATA&extensionSet.extensionNames=HANGOUTS_OFF_NETWORK_GAIA_GET&extensionSet.extensionNames=HANGOUTS_PHONE_DATA&includedProfileStates=ADMIN_BLOCKED&includedProfileStates=DELETED&includedProfileStates=PRIVATE_PROFILE&mergedPersonSourceOptions.includeAffinity=CHAT_AUTOCOMPLETE&coreIdParams.useRealtimeNotificationExpandedAcls=apt&requestMask.includeField.paths=person.email&requestMask.includeField.paths=person.gender&requestMask.includeField.paths=person.in_app_reachability&requestMask.includeField.paths=person.metadata&requestMask.includeField.paths=person.title&requestMask.includeField.paths=person.phone&requestMask.includeField.paths=person.portray&requestMask.includeField.paths=person.read_only_profile_info&requestMask.includeField.paths=person.group&requestMask.includeField.paths=person.hassle&requestMask.includeField.paths=person.cover_photo&requestMask.includeContainer=PROFILE&requestMask.includeContainer=DOMAIN_PROFILE&requestMask.includeContainer=CONTACT&key=REDACTED'

So I apt want 4 headers: Authorization, Divulge material-Kind, Foundation, and Cookie. That’s great extra manageable.

step 4: translate it into Python

Now that every person is conscious of what headers we would like, we are able to translate our curl expose into a Python program!
This part would per chance even be a lovely mechanical project, the aim is apt to ship the very same files with Python as we were with curl.

Here’s what that looks delight in. Here is precisely similar to the outdated curl
expose, but utilizing Python’s requests. I also broke up the very long ask body
string into an array of tuples to originate it less difficult to work with
programmmatically.

import requests
import urllib

files=[
    ('personId','101777723'), # I redacted these IDs a bit too
    ('personId','117533904'),
    ('personId','111526653'),
    ('personId','116731406'),
    ('extensionSet.extensionNames','HANGOUTS_ADDITIONAL_DATA'),
    ('extensionSet.extensionNames','HANGOUTS_OFF_NETWORK_GAIA_GET'),
    ('extensionSet.extensionNames','HANGOUTS_PHONE_DATA'),
    ('includedProfileStates','ADMIN_BLOCKED'),
    ('includedProfileStates','DELETED'),
    ('includedProfileStates','PRIVATE_PROFILE'),
    ('mergedPersonSourceOptions.includeAffinity','CHAT_AUTOCOMPLETE'),
    ('coreIdParams.useRealtimeNotificationExpandedAcls','true'),
    ('requestMask.includeField.paths','person.email'),
    ('requestMask.includeField.paths','person.gender'),
    ('requestMask.includeField.paths','person.in_app_reachability'),
    ('requestMask.includeField.paths','person.metadata'),
    ('requestMask.includeField.paths','person.name'),
    ('requestMask.includeField.paths','person.phone'),
    ('requestMask.includeField.paths','person.photo'),
    ('requestMask.includeField.paths','person.read_only_profile_info'),
    ('requestMask.includeField.paths','person.organization'),
    ('requestMask.includeField.paths','person.location'),
    ('requestMask.includeField.paths','person.cover_photo'),
    ('requestMask.includeContainer','PROFILE'),
    ('requestMask.includeContainer','DOMAIN_PROFILE'),
    ('requestMask.includeContainer','CONTACT'),
    ('key','REDACTED')
]
response=requests.post('https://folks-pa.clients6.google.com/v2/folks/?key=REDACTED',
    headers={
        'X-HTTP-Contrivance-Override': 'GET',
        'Authorization': 'SAPISIDHASH REDACTED',
        'Divulge material-Kind': 'application/x-www-develop-urlencoded',
        'Foundation': 'https://hangouts.google.com',
        'Cookie': 'REDACTED',
    },
    files=urllib.parse.urlencode(files),
)

print(response.text)

I ran this program and it truly works – it prints out a bunch of JSON! Hooray!

You’ll see that I replaced a bunch of issues with REDACTED, that’s because
if I included these values that you would be capable to access the Google Hangouts API for my
yarn which will seemingly be no apt.

and we’re done!

Now I will alter the Python program to conclude whatever I’d like, delight in passing
assorted parameters or parsing the output.

I’m now not going to conclude the leisure attention-grabbing with it because I’m now not truly
drawn to utilizing this API in any appreciate, I apt wished to show veil what the project appears to be like delight in.

Nonetheless we gain help a bunch of JSON that you would be capable to indubitably conclude one thing with.

curlconverter appears to be like immense

Any individual commented that you doubtlessly can translate curl to Python (and a bunch of other
languages!) routinely with https://curlconverter.com/ which appears to be like improbable
– I’ve incessantly done it manually. I attempted it out on this case and it looks
to work immense.

determining how the API works is nontrivial

I don’t are wanting to undersell how now not easy it is going to even be to desire out how an unknown
API works – it’s now not evident! I establish now not want any conception what moderately a few the parameters to
this Google Hangouts API conclude!

Nonetheless moderately a few the time there are some parameters that appear gorgeous straightforward,
delight in requestMask.includeField.paths=person.email doubtlessly manner “contain every
person’s email tackle”. So I are attempting to point of interest on the parameters I conclude trace
better than these I don’t trace.

this incessantly works (in conception)

About a of you would be questioning – are you able to largely conclude this?

The resolution is form of yes – browsers aren’t magic! All the records
browsers ship to your backend is apt HTTP requests. So if I replica the total
HTTP headers that my browser is sending, I deem there’s actually no manner for
the backend to tell that the ask isn’t despatched by my browser and is truly
being despatched by a random Python program.

Take into accout the truth that, we removed a bunch of the headers the browser despatched so theoretically
the backend would per chance tell, but in general they obtained’t test.

There are some caveats even though – shall we embrace moderately a few Google services enjoy
backends that communicate with the frontend in a truly inscrutable (to me)
manner, so even supposing in conception that you would be capable to mimic what they’re doing, in note
it will seemingly be almost now not doable. And bigger APIs that approach across extra abuse
would per chance enjoy extra protections.

Now that we’ve viewed how one can expend undocumented APIs delight in this, let’s talk about
some issues that would per chance stagger injurious.

anguish 1: expiring session cookies

One big anguish right here is that I’m utilizing my Google session cookie for
authentication, so this script will cease working at any time when my browser session
expires.

That manner that this way wouldn’t work for a long running program (I’d
are wanting to make expend of an exact API), but after I apt must instant rep a miniature little bit of files as a
1-time thing, it is going to work immense!

anguish 2: abuse

If I’m utilizing a dinky online page, there’s a gigantic gamble that my miniature Python script
would per chance keep down their provider because it’s doing manner extra requests than they’re
ready to tackle. So after I’m doing this I are attempting to be respectful and now not originate too
many requests too instant.

Here is especially main because moderately a few web sites which don’t enjoy pleasant
APIs are smaller web sites with less resources.

In this case clearly this isn’t a disaster – I deem I made 20 requests
complete to the Google Hangouts backend whereas penning this weblog post, which they
can indubitably kind out.

Moreover if you’re utilizing your yarn credentials to access the API in a excessive
manner and you space off concerns, that you would be capable to (very moderately) gain your yarn
suspended.

I also follow downloading files that’s either mine or that’s intended to be
publicly accessible – I’m now not wanting for vulnerabilities.

be conscious that anybody can expend your undocumented APIs

I deem the largest thing to clutch about this isn’t truly how one can expend other
folks’s undocumented APIs. It’s fun to conclude, but it has lots
of boundaries and I don’t truly conclude it that in general.

It’s great extra main to trace that anybody can conclude this to your
backend API! Everyone has developer instruments and the community tab, and it’s gorgeous
easy to secret agent which parameters you’re passing to the backend and to switch them.

So if anybody can apt switch some parameters to gain yet one more person’s files,
that’s no apt. I deem most developers constructing publicly availble APIs know
this, but I’m pointing out it because each person needs to be taught it for the predominant
time ultimately 🙂