Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
menu search
person
editAsk a Question
Welcome To Ask or Share your Answers For Others

Categories

retrofit - Java Okhttp3 "Please enable JS and disable any ad blocker"

in Technique[技术] by (71.8m points)

I am trying to get a request to get the json file but once i do that I get the Log that i get is that it doesn't see the json file. infact, it only sees the Log below.

Should i get another response than this in order to get the info of the Json file?

This is the link i want to get a request from: https://www.footlocker.nl/INTERSHOP/static/FLE/Footlocker-Site/-/Footlocker/%22%20+%20%22en_US/Release-Calendar/Launchcalendar/launchdata/launchcalendar_feed_all.json

jan. 10, 2021 11:26:59 P.M. okhttp3.internal.platform.Platform log
INFO: <html><head><title>footlocker.nl</title><style>#cmsg{animation: A 1.5s;}@keyframes A{0%{opacity:0;}99%{opacity:0;}100%{opacity:1;}}</style></head><body style="margin:0"><p id="cmsg">Please enable JS and disable any ad blocker</p><script>var dd={'cid':'AHrlqAAAAAMA_bz9MUAng5wAUHCDZw==','hsh':'A55FBF4311ED6F1BF9911EB71931D5','t':'fe','s':17434,'host':'geo.captcha-delivery.com'}</script><script src="https://ct.captcha-delivery.com/c.js"></script></body></html>

And this is my code:

public FootlockerV2() {
    tester();
}

public void tester() {
    
    var loggingInterceptor = new HttpLoggingInterceptor();
    loggingInterceptor.setLevel(HttpLoggingInterceptor.Level.BODY);

    //httpClient maken.
    OkHttpClient.Builder httpClient = new OkHttpClient.Builder();

    httpClient.addInterceptor(chain -> {

        Request request = chain.request();
        Request.Builder newRequest = request.newBuilder()
                .header("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 " +
                        "(KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36")
                .header("authority", "www.footlocker.nl")
                .header("scheme", "https")
                .header("cache-control", "no-cache")
                .header("path", "/INTERSHOP/static/FLE/Footlocker-Site/-/Footlocker/%22%20+%20%22en_US/Release-Calendar/Launchcalendar/launchdata/launchcalendar_feed_all.json")
                .header("accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9");

        return chain.proceed(newRequest.build());
    }).addInterceptor(loggingInterceptor);

    Retrofit retrofit = new Retrofit.Builder()

            .client(httpClient.build())
            .baseUrl(Api.base)
            .addConverterFactory(GsonConverterFactory.create())
            .build();

    Api api = retrofit.create(Api.class);


    Call<List<Article>> call = api.getArticles();

    call.enqueue(new Callback<List<Article>>() {
        @Override
        public void onResponse(Call<List<Article>> call, Response<List<Article>> response) {


            List<Article> articles = response.body();

            if (articles != null) {
                for (Article article : articles) {

                    System.out.println(response.body().toString());
                    System.out.println(article.getName() + "
");
                }
            }


        }

        @Override
        public void onFailure(Call<List<Article>> call, Throwable t) {

        }
    });

}`

与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
1.5k views
Welcome To Ask or Share your Answers For Others

1 Answer

by (71.8m points)

Dealing with such response is more or less trivial, and usually follows the following pattern:

Copy the generated curl command

Open the network request in the debugger Network tab (assuming you're on Google Chrome/Chromium if I've parsed the User-Agent header properly), and copy it using the "Copy as cURL" menu item. Once it's copied, double-check the generated command works well in the terminal.

Trim off unnecessary parameters from the curl command

I usually create a quick small git repo for such experiments, where I commit every successful step, and first put the command to an executable bash script to run it quickly. The more you trim, the smaller the curl command becomes, so that you have as small curl command as possible. Here is what I got:

#!/bin/bash

curl 'https://www.footlocker.nl/INTERSHOP/static/FLE/Footlocker-Site/-/Footlocker/%22%20+%20%22en_US/Release-Calendar/Launchcalendar/launchdata/launchcalendar_feed_all.json' 
    -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36' 
    -H 'accept-language: en-US,en;q=0.9' 
    --compressed

This is the smallest viable command for your request. Now enable the -v switch so that you could inspect what curl actually sends to the server:

#!/bin/bash

curl -v 'https://www.footlocker.nl/INTERSHOP/static/FLE/Footlocker-Site/-/Footlocker/%22%20+%20%22en_US/Release-Calendar/Launchcalendar/launchdata/launchcalendar_feed_all.json' 
    -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36' 
    -H 'accept-language: en-US,en;q=0.9' 
    --compressed

You could see something like:

> GET /INTERSHOP/static/FLE/Footlocker-Site/-/Footlocker/%22%20+%20%22en_US/Release-Calendar/Launchcalendar/launchdata/launchcalendar_feed_all.json HTTP/2
> Host: www.footlocker.nl
> Accept: */*
> Accept-Encoding: deflate, gzip
> user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36
> accept-language: en-US,en;q=0.9

Since you seem to use a Windows system, you don't necessarily need bash, cmd should be fine, but make sure Chrome/Chromium generates a cmd-compatible curl command.

Translate the curl request pattern to the Retrofit interceptor

Note that it does not work without --compressed that is actually mapped to Accept-Encoding: deflate, gzip.

okhttp 3: how to decompress gzip/deflate response manually using Java/Android

final class GunzipRequestInterceptor
        implements Interceptor {

    private static final Interceptor instance = new GunzipRequestInterceptor();

    private GunzipRequestInterceptor() {
    }

    static Interceptor getInstance() {
        return instance;
    }

    @Override
    public Response intercept(final Chain chain)
            throws IOException {
        return gunzip(chain.proceed(chain.request()));
    }

    private static Response gunzip(final Response response) {
        if ( response.body() == null ) {
            return response;
        }
        final String contentEncoding = response.headers().get("Content-Encoding");
        if ( !contentEncoding.equals("gzip") ) {
            return response;
        }
        final Headers headers = response.headers().newBuilder().build();
        final ResponseBody body = new RealResponseBody(headers, Okio.buffer(new GzipSource(response.body().source())));
        return response.newBuilder()
                .headers(headers)
                .body(body)
                .build();
    }

}

interface IFootLockerClient {

    @GET("/INTERSHOP/static/FLE/Footlocker-Site/-/Footlocker/"%20+%20"en_US/Release-Calendar/Launchcalendar/launchdata/launchcalendar_feed_all.json")
    Call<JsonElement> get();

}

final Retrofit retrofit = new Retrofit.Builder()
        .baseUrl("https://www.footlocker.nl/")
        .addConverterFactory(GsonConverterFactory.create())
        .client(new OkHttpClient.Builder()
                .addInterceptor(GunzipRequestInterceptor.getInstance())
                .addInterceptor(chain -> chain.proceed(chain.request()
                        .newBuilder()
                        .header("Accept", "*/*")
                        .header("Accept-Encoding", "gzip")
                        .header("Accept-Language", "en-US,en;q=0.9")
                        .header("User-Agent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36")
                        .build()
                ))
                .build()
        )
        .build();
final IFootLockerClient client = retrofit.create(IFootLockerClient.class);
final Call<JsonElement> jsonElement = client.get();
System.out.println(jsonElement.execute().body());

This should work and write a well-parsed JSON response root element to stdout.


与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share

Just Browsing Browsing

548k questions

547k answers

4 comments

86.3k users

Most popular tags

Theme made by Momin Raza, Modified by Ostack
Powered by Question2Answer
...