I don't think you need to create a new dictionary for that.

Just directly slice your data frame whenever needed.

df[['name', 'time', 'session 1']]

If you think the following design can help you, you can set the name and time to be indexes (df.set_index(['name', 'time'])) and just simply

df['session 1']

import pandas as pd

# Create DataFrame
data = [{'analyte': 'sample1'},
        {'analyte': 'bacon', 'CAS1': 1},
        {'analyte': 'eggs', 'CAS1': 2},
        {'analyte': 'money', 'CAS1': 3, 'CAS2': 1, 'Value2': 1.11},
        {'analyte': 'shoe', 'CAS1': 4},
        {'analyte': 'boy', 'CAS1': 5},
        {'analyte': 'girl', 'CAS1': 6},
        {'analyte': 'onion', 'CAS1': 7, 'CAS2': 4, 'Value2': 6.53},
        {'analyte': 'sample2'},
        {'analyte': 'bacon', 'CAS1': 1},
        {'analyte': 'eggs', 'CAS1': 2, 'CAS2': 1, 'Value2': 7.88},
        {'analyte': 'money', 'CAS1': 3},
        {'analyte': 'shoe', 'CAS1': 4, 'CAS2': 3, 'Value2': 15.5},
        {'analyte': 'boy', 'CAS1': 5},
        {'analyte': 'girl', 'CAS1': 6},
        {'analyte': 'onion', 'CAS1': 7}]
df = pd.DataFrame(data)

# Create list of row indices for each sample name
# For above example: s = [0, 8, 16]
s = list(df['analyte'].index[df['analyte'].str[:6] == 'sample']) + [len(df)]

# Create new dataframes for each sample and print results
samples = {}
for i, j in zip(s, s[1:]):
    sample_df = df.iloc[i+1 : j, :].reset_index(drop=True)
    sample_name = df.iloc[i].loc['analyte']
    samples.update( {sample_name : sample_df} )

print(samples['sample2'])

Other options:

# if CAS1 cell of sample row is NaN
sample_indices = list(df['CAS1'].index[df['CAS1'].apply(np.isnan)]) + [len(df)]

# if CAS1 cell of sample row is NaN or None
sample_indices = list(df['CAS1'].index[df['CAS1'].isnull()]) + [len(df)]

# if CAS1 cell of sample row is an empty string
sample_indices = list(df['CAS1'].index[df['CAS1'] == '']) + [len(df)]

You can use df.columns to get all column names and then create sub-dataframes:

outdflist =[]
# for each column beyond first: 
for col in oridf.columns[1:]:
    # create a subdf with desired columns:
    subdf = oridf[['GeneID',col]]
    # append subdf to list of df: 
    outdflist.append(subdf)

# to view all dataframes created: 
for df in outdflist:
    print(df)

Output:

   GeneID    A
0       1  0.3
1       2  0.5
2       3  0.4
3       4  0.9
4       5  0.1
   GeneID    B
0       1  0.2
1       2  0.3
2       3  0.1
3       4  0.7
4       5  0.4
   GeneID    C
0       1  0.6
1       2  0.1
2       3  0.5
3       4  0.1
4       5  0.7
   GeneID    D
0       1  0.4
1       2  0.2
2       3  0.1
3       4  0.6
4       5  0.2
   GeneID    E
0       1  0.8
1       2  0.6
2       3  0.3
3       4  0.7
4       5  0.5

Above for loop can also be written more simply as list comprehension:

outdflist = [ oridf[['GeneID', col]] 
              for col in oridf.columns[1:] ]

Just to underline my comment to @maxymoo's answer, it's almost invariably a bad idea ("code smell") to add names dynamically to a Python namespace. There are a number of reasons, the most salient being:

1. Created names might easily conflict with variables already used by your logic.

2. Since the names are dynamically created, you typically also end up using dynamic techniques to retrieve the data.

This is why dicts were included in the language. The correct way to proceed is:

d = {}
for name in companies:
    d[name] = pd.DataFrame()

Nowadays you can write a single dict comprehension expression to do the same thing, but some people find it less readable:

d = {name: pd.DataFrame() for name in companies}

Once d is created the DataFrame for company x can be retrieved as d[x], so you can look up a specific company quite easily. To operate on all companies you would typically use a loop like:

for name, df in d.items():
    # operate on DataFrame 'df' for company 'name'

In Python 2 you were better writing

for name, df in d.iteritems():

because this avoids instantiating the list of (name, df) tuples that .items() creates in the older version. That's now largely of historical interest, though there will of course be Python 2 applications still extant and requiring (hopefully occasional) maintenance.

You could certainly use a variable instead of hard-coding the ticker:

ticker = Equity_Tickers[0]  # FB
data = pdr.DataReader(ticker, 'yahoo', start, end)

To create a loop, you could do something like:

for ticker in Equity_Tickers:
    data = pdr.DataReader(ticker, 'yahoo', start, end)
    df = data[['Adj Close']]

The tricky thing is storing the df variable in each iteration of the loop instead of overwriting it. One solution is to store each DataFrame in a list or dictionary.

frames = []
for ticker in Equity_Tickers:
    data = pdr.DataReader(ticker, 'yahoo', start, end)
    frames.append(data[['Adj Close']])

# eg, use frames[0] to access first ticker's DataFrame

OR

frames = {}
for ticker in Equity_Tickers:
    data = pdr.DataReader(ticker, 'yahoo', start, end)
    frames[ticker] = data[['Adj Close']]

# eg, use frames['FB'] to access FB ticker's DataFrame

If you're feeling really fancy you could also use a list comprehension / dictionary comprehension to accomplish this in one line:

# list comprehension
frames = [pdr.DataReader(t, 'yahoo', start, end)[['Adj Close']] for t in Equity_Tickers]

# dict comprehension
frames = {t: pdr.DataReader(t, 'yahoo', start, end)[['Adj Close']] for t in Equity_Tickers}

Additional note: You might also consider combining the frames into one large DataFrame, perhaps a tidy-style DataFrame where "ticker" is a field for each row.

Unsure if your saving your variable into a csv or keep it in memory for further use,

you could pass each unique value into a dict and access by it's value :

    print(df)
     Cal  Dates
0    85     23
1    75     23
2    74     23
3    97     23
4    54     24
5    10     24
6    77     24
7    95     24
8    58     25
9    53     25
10   44     25
11   94     25



d = {}

for frame, data in df.groupby('Dates'):
    d[f'df{frame}'] = data


print(d['df23'])
    Cal  Dates
0   85     23
1   75     23
2   74     23
3   97     23

edit updated request :

for k,v in d.items():
    i = (v['Cal'].loc[v['Cal'] > 70].count())
    print(f"{v['Dates'].unique()[0]} --> {i} times")
23 --> 4 times
24 --> 2 times
25 --> 1 times

You're definitely along the right lines that you want a for loop. Also this merged_tables list looks hopeful. if you put all the different parameters into lists, you can do something like this:

names = ('me', 'you')
nums = (6, 9)
tables = []
for name, num in zip(names, nums):
    tables.append(make_table(name, num)

Whenever you start naming variables thing1, thing2, thing3... It's a pretty sure sign you should put them into a list or dictionary.

I am not sure if you can create the names of dataframes dynamically in PySpark. In Python, you cannot even dynamically assign the names of variables, let alone dataframes.

One way is to create a dictionary of the dataframes, where the key corresponds to each date and the value of that dictionary corresponds to the dataframe.

For Python: Refer to this link, where someone has asked a similar Q on name dynamism.

Here is a small PySpark implementation -

from pyspark.sql.functions import col
values = [('2018-01-01','M',100),('2018-02-01','F',100),('2018-03-01','M',100)]
df = sqlContext.createDataFrame(values,['date','gender','balance'])
df.show()
+----------+------+-------+
|      date|gender|balance|
+----------+------+-------+
|2018-01-01|     M|    100|
|2018-02-01|     F|    100|
|2018-03-01|     M|    100|
+----------+------+-------+

# Creating a dictionary to store the dataframes.
# Key: It contains the date from my_list.
# Value: Contains the corresponding dataframe.
dictionary_df = {}  

my_list = ['2018-01-01', '2018-02-01', '2018-03-01']
for i in my_list:
    dictionary_df[i] = df.filter(col('date')==i)

for i in my_list:
    print('DF: '+i)
    dictionary_df[i].show() 

DF: 2018-01-01
+----------+------+-------+
|      date|gender|balance|
+----------+------+-------+
|2018-01-01|     M|    100|
+----------+------+-------+

DF: 2018-02-01
+----------+------+-------+
|      date|gender|balance|
+----------+------+-------+
|2018-02-01|     F|    100|
+----------+------+-------+

DF: 2018-03-01
+----------+------+-------+
|      date|gender|balance|
+----------+------+-------+
|2018-03-01|     M|    100|
+----------+------+-------+

print(dictionary_df)
    {'2018-01-01': DataFrame[date: string, gender: string, balance: bigint], '2018-02-01': DataFrame[date: string, gender: string, balance: bigint], '2018-03-01': DataFrame[date: string, gender: string, balance: bigint]}

To solve the error, array 1 which is a multidimensional array is manipulated so as to get each array, this is done by converting it to a list.

Then a dictionary with all array values is created to make the solution elegant.

import pandas as pd

# get number of columns in multidimensional array 
num_cols = len(array1)

# create column names
array1_col_names = ['reps_' + str(i) for i in range(num_cols)]

# get elements of array 1 and store them in a dictionary
values = dict(zip(col_names, list(array1)))
values.update({'labels': array2, 'weight': array3}) # add other arrays to dictionary

# create dataframe from the dictionary
df = pd.DataFrame(values)

df

You can use concat:

In [11]: pd.concat([df1['c'], df2['c']], axis=1, keys=['df1', 'df2'])
Out[11]: 
                 df1       df2
2014-01-01       NaN -0.978535
2014-01-02 -0.106510 -0.519239
2014-01-03 -0.846100 -0.313153
2014-01-04 -0.014253 -1.040702
2014-01-05  0.315156 -0.329967
2014-01-06 -0.510577 -0.940901
2014-01-07       NaN -0.024608
2014-01-08       NaN -1.791899

[8 rows x 2 columns]

The axis argument determines the way the DataFrames are stacked:

df1 = pd.DataFrame([1, 2, 3])
df2 = pd.DataFrame(['a', 'b', 'c'])

pd.concat([df1, df2], axis=0)
   0
0  1
1  2
2  3
0  a
1  b
2  c

pd.concat([df1, df2], axis=1)

   0  0
0  1  a
1  2  b
2  3  c

I suggest use str.get_dummies:

df = df.join(df.pop('Fruit_Color').str.get_dummies(','))
print (df)
  Fruit_Type Fruit_Description  Green  Red  Yellow
0      Apple     Just an apple      1    1       1
1     Banana     Just a Banana      1    0       1
2     Orange    Just an Orange      0    1       1
3      Grape      Just a Grape      0    0       0

You can use pd.read_csv with the correct separators, but you have to do 2 things:

1. Before: Split the parts of the file for Level1 and Level2
2. After: Transpose and set the columns of Level1

Here's the code, straight inside your with open [...] line

reader = output.read() # simply the entire file text, not split into lines
parts = reader.split('[Level2]\n')
lvl1_lines = parts[0].split('[Level1]\n')[1].replace('"','')
lvl2_lines = "Speed   Energylevel\n" + parts[1]

from io import StringIO # to read strings as files for read_csv
df_level1 = pd.read_csv(StringIO(lvl1_lines), sep='\s*=\s*', engine='python').transpose().reset_index()
df_level1.columns = df_level1.iloc[0] # set the correct column names
df_level1 = df_level1[1:] # remove the column row
df_level2 = pd.read_csv(StringIO(lvl2_lines), sep='\\s+')

It would be better if you use a dict, also you can directly pass a url to pandas.read_csv. So the simplified code would look like this:

import pandas as pd

periods = ['2012-1st-quarter','2012-2nd-quarter', '2012-3rd-quarter', '2012-4th-quarter']
url = 'https://www.capitalbikeshare.com/assets/files/trip-history-data/{}.csv'
d = {period: pd.read_csv(url.format(period)) for period in periods}

Then you can access a specific DataFrame like this:

 d['2012-4th-quarter']

To iterate through all Dataframes:

for period, df in d.items():
    print period
    print df