You can unset session variable using:

1. session_unset - Frees all session variables (It is equal to using: $_SESSION = array(); for older deprecated code)
2. unset($_SESSION['Products']); - Unset only Products index in session variable. (Remember: You have to use like a function, not as you used)
3. session_destroy — Destroys all data registered to a session

To know the difference between using session_unset and session_destroy, read this SO answer. That helps.

Unset will destroy a particular session variable whereas session_destroy() will destroy all the session data for that user.

It really depends on your application as to which one you should use. Just keep the above in mind.

unset($_SESSION['name']); // will delete just the name data

session_destroy(); // will delete ALL data associated with that user.

Try using wp_doing_ajax() like so:

function unset_filter_session() {
    if ( ! wp_doing_ajax() ) {
        //Reset sessions on refresh page
        unset( $_SESSION['expenditure_filter'] );
    }
}

UPDATE

You can check the answer's revision for this update part..

UPDATE #2

Sorry, I didn't realize that you're loading a page fragment (#content-area) using the jQuery.load() method. Or that you're not using the admin-ajax.php to handle the AJAX request.

So if you're not using the wp_ajax_{action} or wp_ajax_nopriv_{action} action, or with the way you do the AJAX request, you can check whether the X-Requested-With header is set and that its value is XMLHttpRequest, and if so, you can cancel the session unsetting, like so:

function unset_filter_session() {
    if ( empty( $_SERVER['HTTP_X_REQUESTED_WITH'] ) ||
        'XMLHttpRequest' !== $_SERVER['HTTP_X_REQUESTED_WITH'] ) {
        //Reset sessions on refresh page, if not doing AJAX request
        unset( $_SESSION['expenditure_filter'] );
    }
}

That should work because jQuery always sets that header, unless you explicitly remove it — or change its value.

See the headers option on the jQuery.ajax() reference:

The header X-Requested-With: XMLHttpRequest is always added, but its default XMLHttpRequest value can be changed

Note: The header name is X-Requested-With, but in the superglobal $_SERVER, its key is HTTP_X_REQUESTED_WITH. I.e. Don't use $_SERVER['X-Requested-With'].

UPDATE #3

As suggested by Lawrence Johnson, you (ahem, we) should use filter_input():

function unset_filter_session() {
    if ( 'XMLHttpRequest' !== filter_input( INPUT_SERVER, 'HTTP_X_REQUESTED_WITH' ) ) {
        //Reset sessions on refresh page, if not doing AJAX request
        unset( $_SESSION['expenditure_filter'] );
    }
}

(but I kept the previous code for reference)