In my experiments on large numeric data, Pandas is consistently 20 TIMES SLOWER than Numpy. This is a huge difference, given that only simple arithmetic operations were performed: slicing of a column, mean(), searchsorted() - see below. Initially, I thought Pandas was based on numpy, or at least its implementation was C optimized just like numpy's. These assumptions turn out to be false, though, given the huge performance gap.

In examples below, data is a pandas frame with 8M rows and 3 columns (int32, float32, float32), without NaN values, column #0 (time) is sorted. data_np was created as data.values.astype('float32'). Results on Python 3.8, Ubuntu:

A. Column slices and mean():

# Pandas 
%%timeit 
x = data.x 
for k in range(100): x[100000:100001+k*100].mean() 

15.8 ms ± 101 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

# Numpy
%%timeit 
for k in range(100): data_np[100000:100001+k*100,1].mean() 

874 µs ± 4.34 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Pandas is 18 times slower than Numpy (15.8ms vs 0.874 ms).

B. Search in a sorted column:

# Pandas
%timeit data.time.searchsorted(1492474643)                                                                                                                                                               
20.4 µs ± 920 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

# Numpy
%timeit data_np[0].searchsorted(1492474643)                                                                                                                                                              
1.03 µs ± 3.55 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

Pandas is 20 times slower than Numpy (20.4µs vs 1.03µs).

EDIT: I implemented a namedarray class that bridges the gap between Pandas and Numpy in that it is based on Numpy's ndarray class and hence performs better than Pandas (typically ~7x faster) and is fully compatible with Numpy'a API and all its operators; but at the same time it keeps column names similar to Pandas' DataFrame, so that manipulating on individual columns is easier. This is a prototype implementation. Unlike Pandas, namedarray does not allow for different data types for columns. The code can be found here: https://github.com/mwojnars/nifty/blob/master/math.py (search "namedarray").

If you want to an answer which tells you to stick with just one type of data structures, here goes one: use pandas series/dataframe structures.

The pandas series object can be seen as an enhanced numpy 1D array and the pandas dataframe can be seen as an enhanced numpy 2D array. The main difference is that pandas series and pandas dataframes has explicit index, while numpy arrays has implicit indexation. So, in any python code that you think to use something like

import numpy as np
a = np.array([1,2,3])

you can just use

import pandas as pd
a = pd.Series([1,2,3])

All the functions and methods from numpy arrays will work with pandas series. In analogy, the same can be done with dataframes and numpy 2D arrays.

A further question you might have can be about the performance differences between a numpy array and pandas series. Here is a post that shows the differences in performance using these two tools: performance of pandas series vs numpy arrays.

Please note that even in an explicit way pandas series has a subtle worse in performance when compared to numpy, you can solve this by just calling the values method on a pandas series:

a.values

The result of apply the values method on a pandas series will be a numpy array!