Videos
Factsheet
/ March 6, 2026; 3 months ago (2026-03-06)
/ March 6, 2026; 3 months ago (2026-03-06)
TensorFlow doesn't have first-class Tensor objects, meaning that there are no notion of Tensor in the underlying graph that's executed by the runtime. Instead the graph consists of op nodes connected to each other, representing operations. An operation allocates memory for its outputs, which are available on endpoints :0, :1, etc, and you can think of each of these endpoints as a Tensor. If you have tensor corresponding to nodename:0 you can fetch its value as sess.run(tensor) or sess.run('nodename:0'). Execution granularity happens at operation level, so the run method will execute op which will compute all of the endpoints, not just the :0 endpoint. It's possible to have an Op node with no outputs (like tf.group) in which case there are no tensors associated with it. It is not possible to have tensors without an underlying Op node.
You can examine what happens in underlying graph by doing something like this
Copytf.reset_default_graph()
value = tf.constant(1)
print(tf.get_default_graph().as_graph_def())
So with tf.constant you get a single operation node, and you can fetch it using sess.run("Const:0") or sess.run(value)
Similarly, value=tf.placeholder(tf.int32) creates a regular node with name Placeholder, and you could feed it as feed_dict={"Placeholder:0":2} or feed_dict={value:2}. You can not feed and fetch a placeholder in the same session.run call, but you can see the result by attaching a tf.identity node on top and fetching that.
For variable
Copytf.reset_default_graph()
value = tf.Variable(tf.ones_initializer()(()))
value2 = value+3
print(tf.get_default_graph().as_graph_def())
You'll see that it creates two nodes Variable and Variable/read, the :0 endpoint is a valid value to fetch on both of these nodes. However Variable:0 has a special ref type meaning it can be used as an input to mutating operations. The result of Python call tf.Variable is a Python Variable object and there's some Python magic to substitute Variable/read:0 or Variable:0 depending on whether mutation is necessary. Since most ops have only 1 endpoint, :0 is dropped. Another example is Queue -- close() method will create a new Close op node which connects to Queue op. To summarize -- operations on python objects like Variable and Queue map to different underlying TensorFlow op nodes depending on usage.
For ops like tf.split or tf.nn.top_k which create nodes with multiple endpoints, Python's session.run call automatically wraps output in tuple or collections.namedtuple of Tensor objects which can be fetched individually.
From the glossary:
A Tensor is a typed multi-dimensional array. For example, a 4-D array of floating point numbers representing a mini-batch of images with dimensions [batch, height, width, channel].
Basically, every data is a Tensor in TensorFlow (hence the name):
- placeholders are Tensors to which you can feed a value (with the
feed_dictargument insess.run()) - Variables are Tensors which you can update (with
var.assign()). Technically speaking,tf.Variableis not a subclass oftf.Tensorthough tf.constantis just the most basic Tensor, which contains a fixed value given when you create it
However, in the graph, every node is an operation, which can have Tensors as inputs or outputs.
I don't think tf.stack is the best option for this simple operation, and indeed we want to avoid tf.Tensor.numpy() to retain compatibility with graph mode.
Instead, you can use tf.convert_to_tensor, which, citing the documentation, accepts Tensor objects, numpy arrays, Python lists, and Python scalars, so most things you would ever want to throw at it:
In [1]: import tensorflow as tf
In [2]: a = tf.constant(3)
In [3]: tf.convert_to_tensor([a, 2])
Out[3]: <tf.Tensor: shape=(2,), dtype=int32, numpy=array([3, 2], dtype=int32)>
This also works in graph mode. Adapting the demonstration from another answer:
In [4]: @tf.function
...: def join_tns_num(tensor, num):
...: return tf.convert_to_tensor([tensor, tf.constant(num)])
...:
In [5]: join_tns_num(a, 42)
Out[5]: <tf.Tensor: shape=(2,), dtype=int32, numpy=array([ 3, 42], dtype=int32)>
So, if you encounter ValueError: TypeError: Scalar tensor has no `len()`, check whether replacing tf.constant by tf.convert_to_tensor is the answer.
You can use tf.stack.
import tensorflow as tf
@tf.function
def join_tns_num(tensor, num):
return tf.stack([tensor, tf.constant(num)], axis=0)
Check function:
>>> join_tns_num(tf.constant(3), 2)
<tf.Tensor: shape=(2,), dtype=int32, numpy=array([3, 2], dtype=int32)>