statistic which can be calculated as the arithmetic mean of the maximum and minimum values of any sample
Midrange stats but this time with more definitions of “midrange”
estimation - Just "take the average" they say. It's not that straightforward, right? - Cross Validated
According to tracking stats, Michael Jordan was arguably both the best slasher and midrange shooter of all time.
Archtype vs Archtype statistics - Bant Midrange
Cloud post has a pretty easy time crushing slow control decks with no wasteland
More on reddit.comVideos
best career shooting percents (minimum 1000 fg) and best midrange seasons for three different midrange definitions i encountered: 10-16ft, 16ft-3p line, 10ft-3p line
This is not a direct answer to your question about loss functions, but I am a Statistician, and I use the jargon of my domain, not the jargon of machine learning. I will attempt to answer the question: "which statistic is the best estimator of the population mean?"
It's incorrect to say generally that the arithmetic mean results in a loss of information. In fact in some circumstances, it can be proven that the arithmetic mean or some function of it contains as much information (Fisher information) as the data themselves. This is the concept of a sufficient statistic, i.e. some summary of the data that is sufficient for the data.
For example, if you know that your data follow a Poisson distribution then the sufficient statistic is $T(X) = X_1 + ... + X_n$. Which is simply the sum of the data. For a Normal distribution where you know the variance then the arithmetic sample mean is the sufficient statistic for the population mean. That is, it contains all of the information and no other statistic will do better. Now granted, we are never in the situation where we know are data are normally distributed and happen to know exactly the variance. But that is why we have the central limit theorem. Even for skewed data, if what you really care about is the population mean, then the arithmetic mean is pretty good bet, especially if you have a lot of observations. So to that end, I would say in a lot of circumstances, especially when you have a lot of observations the arithmetic mean is best if what you care about is the population mean.
Now, if you happen to be in the privileged position of knowing your data come from some other distribution, perhaps some pathological negative exponential distribution, then you're correct there may be a better sufficient statistic. In that circumstance the sufficient statistic for $\mu$ is the minimum observation. This is favorite example of Mukhopadhyay in Probability and Statistical Inference and you will find all exercises you can stomach in there to demonstrate.
To answer your question more generally, about how to choose the best statistic: plot your data. Look at it. Think about where it came from and how it was collected. Think about what it is you are actually trying to make inference on, and whether the way these data were collected is actually appropriate for that. Think about the form your data take: Are they strictly integer data? Proportions with a known denominator? Are they skewed, if so would a log-normal make a for a good approximation? Choose a parametric family that seems to satisfy and caveat if you must.
Piggy-backing / building off of Dalton's answer:
Your question, as posed, is incomplete. "Information" as a statistical concept is only defined with reference to an unknown parameter as well as some statistic (function of the full data)—including the degenerate case of the full data. Exactly no statistic of the data has more information about any parameter than the full data, but sometimes statistics have just as much information as the full data about a specific parameter.
Your intuition seems to be that summary statistics reduce the informational content of the full data, but again, there is no informational content to even the full data unless it is with reference to some parameter. It is true that, if you are estimating the variance of a Normal population, the statistic $S^2$ (sample variance) has no informational content about $\mu$ (the population mean). But as pointed out above, $\bar{X}$ contains equal information about $\mu$ as does the full data.
Your question is incomplete because the intuitive, casual definition of "information" is very different from its mathematical definition. For any specific real-world scenario, you will of course have to judge what is an appropriate assumption for the distribution (accounting for skewness, support, etc) and, consequently, what specific statistics preserve the information you need.
As an aside, $\bar{X}$ gives you no information about the variance of a Normal($\mu$, $\sigma^2$) distribution, either, but knowing the pair $(\bar{X}, S^2)$ provides the same information as the full data about both parameters.