Found a solution: ExchangeFilterFunction.ofResponseProcessor seems to be what I was looking for.

@Configuration
public class WebClientConfig {

    @Bean
    @LoadBalanced
    public WebClient.Builder webClientBuilder(){
        return WebClient
                .builder()
                .filter(ExchangeFilterFunction.ofResponseProcessor(this::renderApiErrorResponse));
    }

    private Mono<ClientResponse> renderApiErrorResponse(ClientResponse clientResponse) {
        if(clientResponse.statusCode().isError()){
            return clientResponse.bodyToMono(ApiErrorResponse.class)
                    .flatMap(apiErrorResponse -> Mono.error(new ResponseStatusException(
                            clientResponse.statusCode(),
                            apiErrorResponse.getMessage()
                    )));
        }
        return Mono.just(clientResponse);
    }

}

The type parameter in the Predicate (first argument to onStatus(...)) is not of type org.springframework.http.HttpStatus, but of type org.springframework.http.HttpStatusCode (which happens to be an interface that the HttpStatus enum implements).

As such, switch the method reference from HttpStatus::is4xxClientError to HttpStatusCode::is4xxClientError, and the error should go away.