Determinants of Super-Schur Functions, Lattice Paths, and Dotted Plane Partitions

Francesco Brenti^{1} Department of Mathematics University of Michigan Ann Arbor, Michigan 48109-1003 U.S.A.

Contents

1. Introduction
2. Invariant digraphs and super-Schur functions
3. Determinants of super-Schur functions
4. The combinatorics of dotted plane partitions
5. Determinants of symmetric functions
6. q-Binomial determinants
7. q-Stirling determinants of the first kind
8. q-Stirling determinants of the second kind

1 Introduction

Let x = (x_{1}, x_{2}, …) and y = (y_{1}, y_{2}, …) be two sequences of independent variables and λ be a partition. We denote by

s_{λ}(x_{1}, x_{2}, … /y_{1}, y_{2}, …)

the super-Schur function corresponding to λ in the variables x and y. These functions arise naturally in the representation theory of Lie superalgebras [12] and were also defined, independently, by Metropolis, Nicoletti, and Rota in [19], under the name of bisymmetric functions. Since then, they have been studied extensively and we refer the reader to [1], [2], or [6] for their definition (they can be defined in several equivalent ways) and further information about them.

1Partially supported by NSF grant # DMS 8903246

The purpose of the present work is to give combinatorial interpretations to the minors of the infinite matrix

S(x, y) =^{def} (s_{(k)}(x_{1}, …, x_{n}/y_{1}, …, y_{n}))_{n,k∈N} .

Our main results (Theorems 3.3 and 4.3) are proved combinatorially using lattice paths and are stated in terms of dotted and diagonal strict plane partitions, respectively. They also have many applications. As special cases we obtain combinatorial interpretations of determinants of homogeneous, elementary, and Hall-Littlewood symmetric functions, Schur’s Q-functions, q-binomial coefficients, and q-Stirling numbers of both kinds. Other applications include the solution of a problem posed by Yahory in [33] and the combinatorial interpretation of a class of symmetric functions first defined, algebraically, by Macdonald in [18]. Many of our results (including all those in §§7 and 8) are new even in the case q = 1. Others are q-analogues of known results. Our main theorem also has several interesting applications to the theory of total positivity. These are treated in [3].

We now collect some definitions, notation and results that will be used in the rest of the paper. We let P =^{def} {1, 2, 3, …} and N =^{def} P ∪ {0}; for a ∈ N we let [a] =^{def} {1, 2, …, a} (where [0] =^{def} ∅). The cardinality of a set A will be denoted by |A|. For i_{1}, …, i_{t} ∈ P we write {i_{1}, i_{2}, …, i_{t}}_{<} if i_{1} < i_{2} < … < i_{t}. For m, n ∈ P with m ≤ n we let [m, n] =^{def} [n] \ [m − 1]. Given a (finite) set S we denote by Π(S) the set of all (set) partitions of S (see, e.g., [28, p.33] for further information about partitions of a set). For n ∈ P, we denote by S_{n} the symmetric group on n elements. Throughout this work, q will denote an indeterminate. For n ∈ P we let [n]_{q} =^{def} 1 + q + q^{2} + … + q^{n−1}, and [0]_{q} =^{def} 0. We will follow [18, Chap.I] for notation and terminology related to partitions and symmetric functions. In particular, given a partition λ, we denote by λ′ its conjugate, by m_{i}(λ) the number of parts of λ that are equal to i, for i ∈ P, and by s_{λ} (respectively e_{λ}, h_{λ}, p_{λ}) the Schur (respectively elementary, complete homogeneous, power sum) symmetric function associated to λ. We will usually identify a partition λ = (λ_{1}, …, λ_{r}) with its diagram {(i, j) ∈ P × P : 1 ≤ i ≤ r, 1 ≤ j ≤ λ_{i}}.

We will follow [26] for notation and terminology regarding plane partitions. However, we will often need to work with more general objects than skew plane partitions, which we now define. Let λ = (λ_{1}, …, λ_{r}) and μ = (μ_{1}, …, μ_{r}) be two partitions such that μ ⊆ λ. A shifted skew tabloid of shape λ \ μ is an array T = (T_{i,j})_{1≤i≤r, μ_{i}+i≤j≤λ_{i}+i−1} of positive integers (we will sometimes use the notation sh(T) = λ \ μ). We call the sequence (T_{1,1+μ_{1}}, …, T_{r,r+μ_{r}}) the coshape of T, and we denote by T_{i} the i-th row of T, for i = 1, …, r. We also let |T | =^{def} ∑_{(i,j)∈sh(T)} T_{i,j}. We call T a shifted skew plane partition if T_{i,j} ≥ T_{i+1,j} and T_{i,j} ≥ T_{i,j+1} whenever both sides of the inequality are defined. We say that T is row strict (respectively, column strict, diagonal strict) if T_{i,j} > T_{i,j+1} (respectively, T_{i,j} > T_{i+1,j}, T_{i,j} > T_{i+1,j+1}) whenever both sides of the inequality are defined. Given a shifted skew plane parti-

tion T as above we let \overline{T} =^{def} (T_{i,j})_{1≤i≤r, μ_{i}+i+1≤j≤λ_{i}+i−1}, T̃ =^{def} (T_{i,j})_{1≤i≤r, μ_{i}+i≤j≤λ_{i}+i−2},

and T̂ =^{def} (T_{i,j} + 1)_{1≤i≤r, μ_{i}+i≤j≤λ_{i}+i} (where T_{i,i+λ_{i}} =^{def} 0, for i = 1, …, r). If μ = ∅

then we call T a shifted plane partition of shape λ. Note that we do not require the parts of λ to be distinct. Note that if both μ and λ have distinct parts then a shifted skew plane partition of shape λ \ μ is just a skew plane partition of shape (λ_{1}, λ_{2} + 1, …, λ_{r} + r − 1) \ (μ_{1}, μ_{2} + 1, …, μ_{r} + r − 1). Also note that a strict Gelfand pattern of length n (as defined, e.g., in [29], or [7]) with given first row, is equivalent to a diagonal strict shifted plane partition of shape (n, n − 1, …, 3, 2, 1) and given coshape. Therefore diagonal strict shifted plane partitions are a natural generalization of strict Gelfand patterns.

Given an infinite matrix M = (M_{n,k})_{n,k∈N} (where M_{n,k} is the entry in the n-th row and k-th column of M) and {n_{1}, …, n_{r}}_{<}, {k_{1}, …, k_{r}}_{<} ⊆ N we let

M ( \begin{matrix}n_{1}, …, n_{r} \\ k_{1}, …, k_{r} \end{matrix} ) =^{def} det [(M_{n_{i},k_{j}})_{1≤i , j≤r}] .

Given an infinite sequence {a_{i}}_{i∈N} we let

\{a_{i}\}_{i∈N} ( \begin{matrix}n_{1}, …, n_{r} \\ k_{1}, …, k_{r} \end{matrix} ) =^{def} A ( \begin{matrix}n_{1}, …, n_{r} \\ k_{1}, …, k_{r} \end{matrix} )

where A =^{def} (a_{n−k})_{n,k∈N} (where a_{i} =^{def} 0 if i < 0).

Let D = (V, A) be a directed graph (or, digraph, for short). We will always assume that D has no loops or multiple edges, so that we can identify the elements of A with ordered pairs (u, v), with u, v ∈ V, u ≠ v. A path in D is a sequence τ = u_{1}, u_{2} … u_{n} of elements of V such that (u_{i}, u_{i+1}) ∈ A for i = 1, …, n − 1, we then say that τ goes from u_{1} to u_{n}. We say that D is locally finite if for every u, v ∈ V there are only a finite number of paths from u to v. Note that this implies that D is acyclic. We say that D is weighted if there is a function w : A → R, where R is some commutative Q-algebra. Let D = (V, A, w) be a locally finite, weighted, digraph. For a path π = u_{0}u_{1} … u_{k} in D we let

w(π) =^{def} ∏_{i=1}^{k} w(u_{i−1}, u_{i}) ,

and for u, v ∈ V we let

P_{D}(u, v) =^{def} ∑_{π} w(π)

where the sum is over all paths π in D going from u to v. We adopt the convention that P_{D}(u, u) =^{def} 1 for all u ∈ V (i.e. there is only one path, the empty path, from u to u and its weight is 1). We will usually omit the subscript D when there is no danger of confusion. Given u =^{def} (u_{1}, …, u_{r}), v =^{def} (v_{1}, …, v_{r}) ∈ V^{r} we let

N(u, v) =^{def} ∑_{(π_{1},...,π_{r})} w(π_{1}, …, π_{r})

where w(π_{1}, …, π_{r}) =^{def} ∏_{i=1}^{r} w(π_{i}) and where the sum is over all r-tuples of paths (π_{1}, …, π_{r}) from u to v (i.e., π_{i} is a path from u_{i} to v_{i}, for i = 1, …, r) that are non-intersecting (i.e., π_{i} and π_{j} have no vertices in common if i ≠ j). We say that u and v are compatible if, for every σ ∈ S_{r} \ {Id}, there are no r-tuples of paths from (u_{1}, …, u_{r}) to (v_{σ(1)}, …, v_{σ(r)}) that are non-intersecting. The following fundamental result was first proved by Lindström in [17] and has later found numerous applications in enumerative combinatorics (see, e.g., [8], [9], [22], [31]). We refer the reader to [9, Corollary 2], [31, Theorem1.2], or [17, Lemma 1] for its proof.

Lemma 1.1 Let D = (V, A, w) be a locally finite, weighted, digraph and u =^{def} (u_{1}, …, u_{n}), v =^{def} (v_{1}, …, v_{n}) ∈ V^{n} be compatible. Then

N(u, v) = det [(P_{D}(u_{i}, v_{j}))_{1≤i, j≤n}] .□

2 Invariant Digraphs and Super-Schur Functions

Let D = (N × N, A, w) be a, locally finite, weighted digraph. We say that D is weakly y-invariant if

P_{D}((0, m) , (n, k)) = P_{D}((0, 0) , (n, k − m))  (1)

for all m, n, k ∈ N with k ≥ m. We say that D is y-invariant if the map S_{y} : D → D defined by

S_{y}((n, k)) =^{def} (n, k + 1)

is an isomorphism between D and S_{y}(D) (as weighted digraphs). The following result follows easily from (1) and Lemma 1.1.

Theorem 2.1 Let D be a locally finite, weighted, digraph on the vertex set N × N. Assume that D is planar and weakly y-invariant. Let {n_{1}, …, n_{r}}_{<}, {k_{1}, …, k_{r}, m}_{<} ⊆ N, and n ∈ N. Then

(P_{D}((0, 0), (n, k)))_{n,k∈N} ( \begin{matrix}n_{1}, …, n_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = N( ((0, m−k_{1}), …, (0, m−k_{r})), ((n_{1}, m), …, (n_{r}, m)) )

(2)

and

\{P_{D}((0, 0), (n, k))\}_{k∈N} ( \begin{matrix}n_{1}, …, n_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = N( ((0, n_{1}), …, (0, n_{r})), ((n, k_{1}), …, (n, k_{r})) ) .□

(3)

We now come to the connection between invariant digraphs and super-Schur functions.

Proposition 2.2 There exists a y-invariant, locally finite, weighted digraph D on N × N, such that

P_{D}((0, 0), (n, k)) = s_{(k)}(x_{1}, …, x_{n}/y_{1}, …, y_{n})

for all (n, k) ∈ N × N.

Proof. We construct D as follows. We put an edge from (n − 1, k) (respectively, (n − 1, k − 1), (n, k − 1)) to (n, k) with weight 1 (respectively, y_{n}, x_{n}) whenever n ∈ P and (n − 1, k) (respectively, (n − 1, k − 1), (n, k − 1)) is in N × N. A portion of D is shown in Figure 1. It is clear that D is y-invariant and locally finite.

[figure: (0,0); (1,0); (0,1); y_{1}; y_{1}; y_{1}; x_{1}; x_{1}; x_{1}; y_{2}; y_{2}; y_{2}; x_{2}; x_{2}; x_{2}; y_{3}; y_{3}; y_{3}; x_{3}; x_{3}; x_{3}; y_{4}; y_{4}; y_{4}; x_{4}; x_{4}; x_{4}; y_{5}; y_{5}; y_{5}; x_{5}; x_{5}; x_{5}]

Figure 1: The digraph D.

Furthermore, it follows from our definition that

P_{D}((0, 0), (n, k)) = x_{n} P_{D}((0, 0), (n, k−1))+y_{n} P_{D}((0, 0), (n−1, k−1))+P_{D}((0, 0), (n−1, k))

(4)

for all n ∈ P, k ∈ N. Therefore

M_{n}(t) = t x_{n}M_{n}(t) + y_{n} t M_{n−1}(t) + M_{n−1}(t)

for n ∈ P, where M_{n}(t) =^{def} ∑_{k≥0} P_{D}((0, 0), (n, k))t^{k}, and hence

M_{n}(t) = ∏_{i=1}^{n} \frac{1 + y_{i} t}{1 − x_{i} t} ,

for n ∈ N. On the other hand, it is well known (see, e.g., [27, p. 271, eq.(9)]) that

s_{(k)}(x_{1}, …, x_{m}/y_{1}, …, y_{n}) = ∑_{j=0}^{k} h_{j}(x_{1}, …, x_{m}) e_{k−j}(y_{1}, …, y_{n}) .  (5)

Hence, using equations (2.2) and (2.5) of Chap.I of [18], we obtain that

∑_{k≥0} s_{(k)}(x_{1}, …, x_{m}/y_{1}, …, y_{n}) t^{k} = ∏_{i=1}^{m}(1 − x_{i} t)^{−1} ∏_{i=1}^{n}(1 + y_{i} t) ,  (6)

and the result follows. □

From now on the letter D will always denote the weighted digraph defined in the proof of the preceding result. Proposition 2.2 and Theorem 2.1 combined already give a combinatorial interpretation for every minor of S(x, y). However, we wish to find a more explicit combinatorial interpretation. This is done in the next section.

3 Determinants of Super-Schur Functions

A dotted partition is a partition λ = (λ_{1}, λ_{2}, …, λ_{n}) where, for each i ∈ P such that m_{i}(λ) > 0, the rightmost occurrence of i in λ may be dotted. So, for example, there are 8 dotted partitions of 3, namely

3 ˙3 21 2˙1 ˙2˙1 ˙21 11˙1 111

In general, there are 2^{d(λ)} ways to dot a partition λ and hence there are ∑_{λ⊢n} 2^{d(λ)} dotted partitions of n, where d(λ) is the number of distinct parts of λ.

The importance of dotted partitions lies in the fact that they offer a very convenient way to encode paths in the digraph D. Given a path τ in D from (a, b) to (c, d) we let, for i = a, … c,

m_{τ}(i) =^{def} min {j : (i, j) ∈ τ} ,

and

M_{τ}(i) =^{def} max {j : (i, j) ∈ τ} .

Given two (possibly) dotted integers we will write a ≐ b to indicate that they are equal as dotted integers, and a = b if they are only equal as integers (so that, for example, 2 = ˙2, 2 ≐ 2, ˙2 ≐ ˙2). We will also write (a + b)^{·} instead of the more

cumbersome \overbrace{a + b}^{·}.

Proposition 3.1 Let m, b, c, d ∈ N with m > b, d. Then there is a bijection between directed paths in D from (0, m − b) to (c, m − d) and dotted partitions into c + 1 parts with smallest part ≐ (d + 1)^{·} and largest part = b + 1.

Proof. Let τ be a directed path in D from (0, m − b) to (c, m − d). For each 0 ≤ i ≤ c let

λ_{i} =^{def} m + 1 − M_{τ}(i) .  (7)

Clearly

b + 1 = λ_{0} ≥ λ_{1} ≥ … ≥ λ_{c−1} ≥ λ_{c} = d + 1 .

Now let j ∈ P be a part of λ_{0}, …, λ_{c} and let i =^{def} max {i : λ_{i} = j}, (so that λ_{i} is the rightmost occurrence of j in λ) then we dot λ_{i} if and only if either i = c or the step of τ leaving (i, M_{τ}(i)) is a diagonal one. This defines a dotted partition satisfying the given conditions. Conversely, let λ = (b + 1 = λ_{0} ≥ λ_{1} ≥ … ≥ λ_{c} ≐ (d + 1)^{·}) be a dotted partition. For i = 1, …, c we let

m_{i} =^{def} \{ \begin{matrix}m + 1 − λ_{i−1} + 1, & if λ_{i−1} is dotted in λ, \\ m + 1 − λ_{i−1}, & otherwise, \end{matrix}  (8)

and m_{0} =^{def} m − b. Note that this implies that

m_{i} ≤ m + 1 − λ_{i}

for i = 0, …, c (since if λ_{i−1} is dotted in λ then λ_{i} < λ_{i−1}). Hence we may define a path τ by

τ =^{def} ⋃_{i=0}^{c}{(i, j) : m_{i} ≤ j ≤ m + 1 − λ_{i}} .

It is then straightforward to verify that τ is indeed a path in D from (0, m − b) to (c, m − d) and that the two maps constructed are inverses of each other and this completes the proof. □

We illustrate the above construction with an example. Suppose d = 0, b = c = 5, m = 7 and let τ be the path from (0, 2) to

[figure: (0,2); (5,7)]

Figure 2.

(5, 7) depicted in Figure 2. Then the corresponding dotted partition is ˙6554˙3˙1.

Given a partition λ = (λ_{1}, …, λ_{r}) a shifted dotted plane partition of shape λ is an array of (possibly dotted) positive integers π = (π_{i,j})_{1≤i≤r, i≤j≤i+λ_{i}−1} where each row is a dotted partition and π_{i,j} > π_{i+1,j} whenever π_{i,j} and π_{i+1,j} are both defined and π_{i+1,j} is not dotted. Note that we do not require the parts of λ to be distinct. For example

5 5 3 3 ˙3 ˙5 ˙3 2 2 ˙2 ˙3 ˙2 1 1

is a shifted dotted plane partition of shape (5, 5, 3, 1). Let π be a shifted dotted plane partition as above. For k = 1, …, λ_{1} we let

t_{k}(π) =^{def} ∑_{i=1}^{d_{k}(π)} π_{i,i+k−1} ,  (9)

where d_{k}(π) =^{def} |{i ∈ P : π_{i,i+k−1} > 0}|, and

ḋ_{k}(π) =^{def} |{i ∈ P : π_{i,i+k−1} is dotted }| .

So d_{k}(π) is the size of the k-th diagonal of π, ḋ_{k}(π) is the number of dots on the k-th diagonal, and t_{k}(π) is the sum of the parts on the k-th diagonal. Note that d_{1}(π) ≥ d_{2}(π) ≥ … ≥ d_{λ_{1}}(π) > 0 is the conjugate partition to λ_{1} ≥ λ_{2} ≥ … ≥ λ_{r} > 0. Also, given π as above we let π̃ =^{def} (π_{ij})_{1≤i≤r; i≤j≤i+λ_{i}−2}. Note that this implies that

d_{i}(π̃) = d_{i+1}(π)  (10)

for all i ∈ N. We also let

t(π) =^{def} (t_{1}(π), …, t_{λ_{1}}(π), 0, 0, …) ,

d(π) =^{def} (d_{1}(π), …, d_{λ_{1}}(π), 0, 0, …) ,

and

ḋ(π) =^{def} (ḋ_{1}(π), …, ḋ_{λ_{1}}(π), 0, 0, …) .

Given a set of variables x = (x_{1}, x_{2}, x_{3}, …) and a vector d = (d_{1}, d_{2}, d_{3}, …) of integers we let

x^{d} =^{def} ∏_{i≥1} x_{i}^{d_{i}} .

and S(d) =^{def} (d_{2}, d_{3}, …). Finally, we define the weight of π to be

w(π) =^{def} y^{ḋ(π̃)} x^{t(π̃)−ḋ(π̃)−S(t(π))} .

For example, if π is

8 ˙8 7 6 ˙5 ˙4 ˙1 ˙6 5 5 4 ˙2 ˙1 ˙5 2 ˙1  (11)

then t(π) = (19, 15, 13, 10, 7, 5, 1, 0, 0, …),

π̃ = 8 ˙8 7 6 ˙5 ˙4 ˙6 5 5 4 ˙2 ˙5 2 ,
d(π̃) = (3, 3, 2, 2, 2, 1, 0, 0, …) ,
ḋ(π̃) = (2, 1, 0, 0, 2, 1, 0, 0, …) ,

and

t(π̃) = (19, 15, 12, 10, 7, 4, 0, 0, …) .

Hence S(t(π)) = (15, 13, 10, 7, 5, 1, 0, 0, …),

t(π̃) − ḋ(π̃) − S(t(π)) = (2, 1, 2, 3, 0, 2, 0, 0, 0, …) ,

and

w(π) = x_{1}^{2} x_{2} x_{3}^{2} x_{4}^{3} x_{6}^{2} y_{1}^{2} y_{2} y_{5}^{2} y_{6} .

Shifted dotted reverse plane partitions (which are defined by reversing the inequalities in every row and column) have already been considered in the literature under the name of shifted P′-tableaux (or shifted circled tableaux) in connection with the projective representations of the symmetric group (see, e.g., [30], [21], [2]) a fact which is not surprising since the super-Schur functions are connected to the characters of these representations (see (29) and the comments following Theorem 5.8). It is interesting to note, however, that they arise here from purely combinatorial considerations.

Theorem 3.2 Let {b_{1}, …, b_{r}}_{>} ⊆ N and c_{1} ≥ … ≥ c_{r}, m, d_{1} ≥ … ≥ d_{r} ∈ N with m > max {b_{1}, d_{1}}, (r ∈ P). Then there is a weight preserving bijection between nonintersecting paths from ((0, m−b_{1}), …, (0, m−b_{r})) to ((c_{1}, m−d_{1}), …, (c_{r}, m−d_{r})) in D and shifted dotted plane partitions of shape (c_{1} +1, …, c_{r} +1) in which the i-th row has smallest part ≐ (d_{i} + 1)^{·} and largest part = b_{i} + 1, for i = 1, …, r.

Proof. Let (τ_{1}, …, τ_{r}) be a set of non-intersecting paths in D from ((0, m − b_{1}), …, (0, m − b_{r})) to ((c_{1}, m − d_{1}), …, (c_{r}, m − d_{r})) (so τ_{i} goes from (0, m − b_{i}) to (c_{i}, m − d_{i}), for i = 1, …, r). By Proposition 3.1 we can associate to τ_{i} a dotted partition λ(τ_{i}) into c_{i} + 1 parts with smallest part ≐ (d_{i} + 1)^{·} and largest part = b_{i} + 1. We form a shifted dotted plane partition π by letting λ(τ_{i}) be the i-th row of it and shifting the resulting array. We only have to check that the conditions on the columns of π are satisfied. For this note that the paths τ_{i} and τ_{i+1} are non-intersecting if and only if

m_{τ_{i+1}}(j) > M_{τ_{i}}(j)  (12)

for each 1 ≤ j ≤ c_{i}. But, by our definitions,

m_{τ_{i+1}}(j) = \{ \begin{matrix}M_{τ_{i+1}}(j − 1) , & if λ_{j−1}(τ_{i+1}) is not dotted, \\ M_{τ_{i+1}}(j − 1) + 1 , & otherwise. \end{matrix}

Hence we conclude from (12) that

M_{τ_{i}}(j) < \{ \begin{matrix}M_{τ_{i+1}}(j − 1) , & if λ_{j−1}(τ_{i+1}) is not dotted, \\ M_{τ_{i+1}}(j − 1) + 1 , & otherwise. \end{matrix}

But, by (7), this means that

λ_{j}(τ_{i}) > \{ \begin{matrix}λ_{j−1}(τ_{i+1}) , & if λ_{j−1}(τ_{i+1}) is not dotted, \\ λ_{j−1}(τ_{i+1}) − 1 , & otherwise, \end{matrix}  (13)

for 1 ≤ j ≤ c_{i}. But, by the definition of π, π_{i j} ≐ λ(τ_{i})_{j−i} for i ≤ j ≤ i + c_{i}, hence (13) is exactly the condition for π to be a dotted plane partition. Hence (τ_{1}, …, τ_{r}) are non-intersecting if and only if π is a dotted plane partition. To prove that this bijection is weight preserving fix 1 ≤ j ≤ r and observe that

w(τ_{j}) = ( ∏_{i∈S_{j}} y_{i}) ∏_{i=1}^{c_{j}} x_{i}^{M_{τ_{j}} (i)−m_{τ_{j}} (i)}  (14)

where S_{j} is the set of those i ∈ [c_{j}] such that the step of τ_{j} leaving (i − 1, M_{τ_{j}}(i − 1)) is a diagonal one. Then, by our definitions we have that S_{j} = {i ∈ [c_{j}] : π_{j,j+i−1} is dotted } and, by ( 7) and (8), there follows that

M_{τ_{j}}(i) − m_{τ_{j}}(i) = \{ \begin{matrix}π_{j,j+i−1} − π_{j,j+i} − 1 , & if i ∈ S_{j}, \\ π_{j,j+i−1} − π_{j,j+i} , & otherwise, \end{matrix}

for i = 1, …, c_{j}. Hence, by (14),

w(τ_{j}) = ∏_{i∈S_{j}} \frac{y_{i}}{x_{i}} ∏_{i=1}^{c_{j}} x_{i}^{π_{j,j+i−1}−π_{j,j+i}} ,

Therefore

w(τ_{1}, …, τ_{r}) = ∏_{j=1}^{r} w(τ_{j})

= ∏_{j=1}^{r} (∏_{i=1}^{c_{j}} x_{i}^{π_{j,j+i−1}−π_{j,j+i}} ∏_{i∈S_{j}} \frac{y_{i}}{x_{i}} )

= ∏_{i=1}^{c_{1}} (x_{i}^{t_{i}(π̃)−t_{i+1}(π)} (\frac{y_{i}}{x_{i}} )^{d_{i}(π̃)}) ,

since

∑_{j=1}^{d_{i}(π̃)}(π_{j,j+i−1} − π_{j,j+i}) = ∑_{j=1}^{d_{i}(π̃)} π_{j,j+i−1} − ∑_{j=1}^{d_{i+1}(π)} π_{j,j+i} = t_{i}(π̃) − t_{i+1}(π) ,

by (9) and (10). Therefore

w(τ_{1}, …, τ_{r}) = ∏_{i≥1} x_{i}^{t_{i}(π̃)−t_{i+1}(π)−ḋ_{i}(π̃)} y_{i}^{ḋ_{i}(π̃)}
= x^{t(π̃)−ḋ(π̃)−S(t(π))} y^{ḋ(π̃)}
= w(π) ,

as desired. □

We illustrate the preceding theorem with an example. Let m = 8 and π be the shifted dotted plane partition given in (11). Then the three non-intersecting paths corresponding to π are those depicted in Figure 3.

[figure: (0,1); (0,3); (0,4); (2,8); (5,8); (6,8)]

Figure 3.

Combining Theorems 3.2 and 2.1 and Proposition 2.2 we obtain the main results of this section.

Theorem 3.3 Let {n_{1}, …, n_{r}}_{>}, {k_{1}, …, k_{r}}_{>} ⊆ N. Then

S(x, y) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∑_{π} x^{t(π̃)−ḋ(π̃)−S(t(π))} y^{ḋ(π̃)}  (15)

where the sum is over all shifted dotted plane partitions π of shape (n_{1}+1, …, n_{r}+1) in which the i-th row has smallest part ≐ ˙1 and largest part = k_{i}+1 for i = 1, …, r. □

Theorem 3.4 Let {m_{1}, …, m_{r}}_{<}, {k_{1}, …, k_{r}}_{<} ⊆ N and m, n ∈ P, m > max{m_{r}, k_{r}}. Then

\{s_{(k)}(x_{1}, …, x_{n}/y_{1}, …, y_{n})\}_{k∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = ∑_{π} x^{t(π̃)−ḋ(π̃)−S(t(π))} y^{ḋ(π̃)} (16)

where the sum is over all shifted dotted plane partitions π of shape ((n + 1)^{r}) in which the i-th row has smallest part ≐ (m − k_{i} + 1)^{·} and largest part = m − m_{i} + 1, for i = 1, …, r. □

Since the minor on the LHS of (16) is just the skew super-Schur function corresponding to the skew shape (m − m_{1} + 1, …, m − m_{r} + r)/(m − k_{1} + 1, …, m − k_{r} + r), Theorem 3.4 gives a combinatorial interpretation for these skew super-Schur functions. Other combinatorial interpretations have been obtained by Berele and Regev [2], Balantekin and Bars [1], Dondi and Jarvis [6], and Stanley [27], (regarding Stanley’s combinatorial interpretation see also Theorem 4.4 of the present work).

4 The combinatorics of dotted plane partitions

In this section we derive some equivalent formulations of Theorems 3.3 and 3.4 which are sometimes more convenient to use. These will be obtained by a combinatorial analysis of shifted dotted plane partitions. First, it will be convenient to introduce some notation.

Let T = (T_{i,j})_{1≤i≤r, i≤j≤i+n_{i}} be a shifted plane partition of shape (n_{1} +1, …, n_{r} + 1). We call a part T_{i,j}, of T, free if T_{i−1,j} > T_{i,j} > T_{i,j+1} (the inequalities being vacuously satisfied if either one of T_{i−1,j} and T_{i,j+1} are undefined). We let

F(T) =^{def} {(i, j) ∈ sh(T) : T_{i,j} is free } ,

and call F(T) the free set of T. Given T as above we define

U(T) =^{def} {(i, j) ∈ sh(T) : (i − 1, j) ∈ sh(T), T_{i−1,j} = T_{i,j}} ,

and call U(T) the upper set of T. We also let

F̃(T) =^{def} {(i, j) ∈ F(T) : j < n_{i} + i} ,

and

Ũ(T) =^{def} {(i, j) ∈ U(T) : j < n_{i} + i} .

Note that, in general, F̃(T) ≠ F(T̃) and Ũ(T) ≠ U(T̃). Given any subset S ⊆ sh(T) and k ∈ P we let

S_{k} =^{def} |{(i, j) ∈ S : j − i + 1 = k}| .

Finally, we let

F (T) =^{def} (F(T)_{1}, F(T)_{2}, …) ,

U(T) =^{def} (U(T)_{1}, U(T)_{2}, …) ,

and define F̃(T), Ũ(T) analogously. For example, if

T = 5 4 4 2 4 3 2 1 2 2

then F (T) = (1, 1, 1, 2, 0, …), U(T) = (1, 1, 1, 0, …), F̃(T) = (1, 1, 1, 0, …), and Ũ(T) = (1, 0, 1, 0, …).

Proposition 4.1 Let k_{1} > … > k_{r}, b_{1} ≥ … ≥ b_{r}, n_{1} ≥ … ≥ n_{r} be positive integers. Then there is a bijection between shifted dotted plane partitions π of shape (n_{1} +1, …, n_{r} +1) in which the i-th row has largest part = k_{i} and smallest part ≐ ḃ_{i}, for i = 1, …, r; and pairs (T, S) where T is a diagonal strict shifted plane partition of shape (n_{1} + 1, …, n_{r} + 1) in which the i-th row has largest part = k_{i} and smallest part = b_{i} for i = 1, …, r, and S ⊆ F̃(T).

Proof. Let π = (π_{i,j})_{1≤i≤r, i≤j≤i+n_{i}} be a shifted dotted plane partition as in the statement of the Proposition. We let the corresponding T be the shifted plane partition obtained by undotting π and

S(π) =^{def} {(i, j) ∈ F̃(T) : π_{i,j} is dotted } .

It is not hard to see that this map π ↦ (T, S(π)) is a bijection between the desired sets. □

Let T = (T_{i,j})_{1≤i≤r, b_{i}+i≤j≤k_{i}+i−1} be a shifted skew plane partition of shape (k_{1}, …, k_{r}) \ (b_{1}, …, b_{r}). For i ∈ P we let c_{i}(T) (respectively r_{i}(T)) be the number of columns (respectively rows) of T that contain at least one part equal to i, and m_{i}(T) be the number of parts of T that are equal to i. We then let

c(T) =^{def} (c_{1}(T), c_{2}(T), …) ,

r(T) =^{def} (r_{1}(T), r_{2}(T), …) ,

and

m(T) =^{def} (m_{1}(T), m_{2}(T), …) .

For example, if T is

4 4 3 3 2 1
5 4 4 2 2
4 2 1
2 1 1

then c(T) = (3, 4, 2, 2, 1, 0, …), r(T) = (3, 4, 1, 3, 1, 0, …), and m(T) = (4, 5, 2, 5, 1, 0, …).

Theorem 4.2 Let k_{1} > … > k_{r}, n_{1} ≥ … ≥ n_{r}, b_{1} ≥ … ≥ b_{r} be positive integers. Then there is a bijection between diagonal strict shifted plane partitions π of shape (n_{1} + 1, …, n_{r} + 1) in which the i-th row has largest part = k_{i} and smallest part = b_{i}, for i = 1, …, r, and diagonal strict shifted skew plane partitions T of shape (k_{1}, …, k_{r}) \ (b_{1}, …, b_{r}) in which the i-th row has largest part ≤ n_{i} for i = 1, … r, and ≥ n_{i+1} + 1 if b_{i} = b_{i+1}, for i = 1, …, r − 1. Furthermore, if π and T correspond under the above bijection then

t(π̃) − S(t(π)) = m(T) ,  (17)
Ũ(π) + F̃(π) = r(T) ,  (18)
Ũ(π) = m(T) − c(T) .  (19)

Proof. Let π be a diagonal strict shifted plane partition satisfying the conditions in the statement of the Theorem. We construct the corresponding T by first conjugating each row of π (so as to keep the resulting array, call it π′, shifted) and then deleting the first b_{i} parts of the i-th row of π′, for i = 1, …, r. It is then not hard to see that the resulting T has all the required properties.

Conversely, given a diagonal strict shifted skew plane partition T = (T_{i,j})_{1≤i≤r, b_{i}+i≤j≤k_{i}+i−1} such that T_{i,i+b_{i}} ≤ n_{i} for i = 1, … r, and T_{i,i+b_{i}} ≥ n_{i+1} + 1 if b_{i} = b_{i+1}, for i = 1, …, r − 1, we construct the corresponding π = (π_{i,j})_{1≤i≤r, i≤j≤i+n_{i}} by first adding b_{i} parts = n_{i} + 1 at the beginning of the i-th row of T, for i = 1, …, r, and then conjugating each row (so as to keep the resulting array shifted). The only property of π that needs some care in order to be verified is diagonal strictness. For this note that it follows from our construction that, for 1 ≤ i ≤ r, i ≤ j ≤ i + n_{i},

π_{i,j} = |{k : T_{i,k} ≥ j − i + 1}| + b_{i} .  (20)

So fix 1 ≤ i ≤ r − 1 and i ≤ j ≤ n_{i+1} + i. We will show that

π_{i,j} > π_{i+1,j+1} .  (21)

Assume first that {k : T_{i+1,k} ≥ j−i+1} ≠ ∅ and let s =^{def} max {k : T_{i+1,k} ≥ j−i+1} (note that this implies that s ≤ k_{i+1} + i ≤ k_{i} + i − 1), then, by (20), π_{i+1,j+1} = s − (b_{i+1} + i) + b_{i+1} = s − i. If s − i < b_{i} then (21) follows; so assume that s − i ≥ b_{i}, then T_{i,s} ≥ T_{i+1,s} ≥ j −i+1, hence |{k : T_{i,k} ≥ j −i+1}| ≥ s−(b_{i} +i−1), therefore π_{i,j} ≥ s − i + 1, and (21) follows. Assume now that {k : T_{i+1,k} ≥ j − i + 1} = ∅. If b_{i} > b_{i+1} then (21) follows; so assume that b_{i} = b_{i+1}, then T_{i,b_{i}+i} ≥ n_{i+1} + 1, hence T_{i,b_{i}+i} ≥ j − i + 1, therefore |{k : T_{i,k} ≥ j − i + 1}| ≥ 1 and (21) follows.

Now let π = (π_{i,j})_{1≤i≤r, i≤j≤i+n_{i}} and T = (T_{i,j})_{1≤i≤r, b_{i}+i≤j≤k_{i}+i−1} correspond under the above bijection. It follows from (20) that, for 1 ≤ i ≤ r and 1 ≤ j ≤ n_{i},

π_{i,j+i−1} − π_{i,j+i} = |{k : T_{i,k} = j}| ,  (22)

and (17) follows. Also, it follows from (22) that π_{i,j+i−1} > π_{i,j+i} if and only if there is at least a part equal to j in the i-th row of T. Hence Ũ(π)_{j} +F̃(π)_{j} = r_{j}(T) and (18) also follows. Finally, let 2 ≤ i ≤ r, i ≤ j ≤ i+n_{i} −1, and max{b_{i−1}, b_{i} +1} ≤ t ≤ k_{i}. We claim that

π_{i,j} = π_{i−1,j} = t ,  (23)

if and only if

T_{i,t+i−1} = T_{i−1,t+i−1} = j − i + 1 .  (24)

In fact, if (23) holds, then from (20) we conclude that T_{i,i+t−1} ≥ j − i + 1, and T_{i−1,t+i−1} ≤ j − i + 1, and (24) follows. Conversely, if (24) holds, then this implies that |{k : π_{i,k} ≥ t}| = |{k : π_{i−1,k} ≥ t + 1}| = j − i + 1, hence π_{i,j} ≥ t, and π_{i−1,j} ≤ t, and (23) follows. This implies that

Ũ(π)_{k} = |{(i, j) ∈ U(T) : T_{i,j} = k}| ,

and (19) follows. □

We illustrate the above bijection with an example. Suppose that

π = 9 8 6 5 4 8 6 5 3 2 6 4 3 2 3 3 1

then

T = 4 3 2 2 1 4 3 3 2 1 1 3 2 1 1 2 2

t(π̃)−S(t(π)) = (26, 21, 14, 8, 0, …)−(21, 15, 10, 6, 0, …) = (5, 6, 4, 2, 0, …) = m(T) ,

Ũ(π) + F̃(π) = (2, 2, 2, 0, …) + (1, 2, 1, 2, 0, …) = (3, 4, 3, 2, 0, …) = r(T) ,

and

m(T) − c(T) = (5, 6, 4, 2, 0, …) − (3, 4, 2, 2, 0, …) = (2, 2, 2, 0, …) = Ũ(π) .

We can now prove the main results of this section.

Theorem 4.3 Let {n_{1}, …, n_{r}}_{>}, {k_{1}, …, k_{r}}_{>} ⊆ N. Then

S(x, y) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∑_{T} y^{m(T)−c(T)} x^{m(T)−r(T)} (y + x)^{r(T)+c(T)−m(T)}

where the sum is over all diagonal strict shifted plane partitions T of shape (k_{1}, …, k_{r}) in which the i-th row has largest part ≤ n_{i} and ≥ n_{i+1} + 1, for i = 1, …, r (where

n_{r+1} =^{def} −1).

Proof. By Theorem 3.3 and Proposition 4.1 we have that

S(x, y) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∑_{T} x^{t(T̃)−S(t(T))} ∑_{S⊆F̃(T)} ∏_{i≥1} (\frac{y_{i}}{x_{i}} )^{Ũ(T)_{i}+S_{i}}
= ∑_{T} y^{Ũ(T)} x^{t(T̃)−S(t(T))−Ũ(T)} ∑_{S⊆F̃(T)} ∏_{i≥1} (\frac{y_{i}}{x_{i}} )^{S_{i}}
= ∑_{T} y^{Ũ(T)} x^{t(T̃)−S(t(T))−Ũ(T)} ∏_{i≥1} (1 + \frac{y_{i}}{x_{i}} )^{F̃(T)_{i}}

where the sum is over all diagonal strict shifted plane partitions T of shape (n_{1} + 1, …, n_{r} + 1) in which the i-th row has smallest part = 1 and largest part = k_{i} + 1, for i = 1, …, r. The thesis now follows from Theorem 4.2.□

Reasoning as in the proof of the preceding theorem but using Theorem 3.4 we obtain the following result.

Theorem 4.4 Let {m_{1}, …, m_{r}}_{<}, {k_{1}, …, k_{r}}_{<} ⊆ N and m, n ∈ P, m > max{m_{r}, k_{r}}. Then

\{s_{(k)}(x_{1}, …, x_{n}/y_{1} …, y_{n})\}_{k∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = ∑_{T} y^{m(T)−c(T)} x^{m(T)−r(T)} (y+x)^{r(T)+c(T)−m(T)}

where the sum is over all diagonal strict plane partitions T of shape (m − m_{1} + 1, …, m − m_{r} + r) \ (m − k_{1} + 1, …, m − k_{r} + r) with largest part ≤ n. □

In the case that k_{i} = i, for i = 1, …, r, the preceding theorem first appeared, though without proof, in [27, Theorem 5.2].

5 Determinants of Symmetric Functions

In this section we consider various determinants of symmetric functions. We begin by considering determinants of elementary symmetric functions. The next two results follow immediately from Theorems 4.3 and 4.4, and the fact that, by (5),

e_{k}(y_{1}, …, y_{n}) = s_{(k)}(0/y_{1}, … y_{n}) .

Corollary 5.1 Let {n_{1}, …, n_{r}}_{>}, {k_{1}, …, k_{r}}_{>} ⊆ N. Then

(e_{k}(y_{1}, …, y_{n}))_{n,k∈N} ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∑_{T} y^{m(\overline{T})}

where the sum is over all row strict shifted plane partitions T of shape (k_{1}+1, …, k_{r}+ 1) and coshape (n_{1} + 1, …, n_{r} + 1). □

Corollary 5.2 Let {m_{1}, …, m_{r}}_{<}, {k_{1}, …, k_{r}}_{<} ⊆ N and m, n ∈ P, m > max{m_{r}, k_{r}}. Then

\{e_{k}(y_{1}, …, y_{n})\}_{k∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = ∑_{T} y^{m(T)}  (25)

where the sum is over all row strict skew plane partitions T of shape (m − m_{1} + 1, …, m − m_{r} + r) \ (m − k_{1} + 1, …, m − k_{r} + r) in which every part is ≤ n. □

The reader will recognize that the above Corollary is the “ conjugate ” Jacobi-Trudi identity (see, e.g., [18, p.40, eq.(5.5)]) since the RHS of (25) is just the Schur function corresponding to the skew shape that is the conjugate of (m − m_{1} + 1, …, m − m_{r} + r) \ (m − k_{1} + 1, …, m − k_{r} + r).

We now consider determinants of complete homogeneous symmetric functions. By (5),

h_{k}(x_{1}, …, x_{n}) = s_{(k)}(x_{1}, …, x_{n}/0) ,

hence the following two results follow immediately from Theorems 4.3 and 4.4, respectively.

Corollary 5.3 Let {n_{1}, …, n_{r}}_{>}, {k_{1}, …, k_{r}}_{>} ⊆ N. Then

(h_{k}(x_{1}, …, x_{n}))_{n,k∈N} ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∑_{T} x^{m(\overline{T})}

where the sum is over all column strict shifted plane partitions T of shape (k_{1} + 1, …, k_{r} + 1) and coshape (n_{1}, …, n_{r}). □

Corollary 5.4 Let {m_{1}, …, m_{r}}_{<}, {k_{1}, …, k_{r}}_{<} ⊆ N and m, n ∈ P, m > max{m_{r}, k_{r}}. Then

\{h_{k}(x_{1}, …, x_{n})\}_{k∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = ∑_{T} x^{m(T)}

where the sum is over all column strict skew plane partitions T of shape (m − m_{1} + 1, …, m − m_{r} + r) \ (m − k_{1} + 1, …, m − k_{r} + r) with largest part ≤ n. □

The preceding result is the well known Jacobi-Trudi identity, (see, e.g., [18, p.40, eq.(5.4)]).

We now want to interpret combinatorially the minors of the matrix (h_{n−k}(x_{1}, …, x_{k}))_{n,k∈N}. For this we will find it convenient to use Theorem 3.3 instead of the more elegant Theorem 4.3.

Theorem 5.5 Let {n_{1}, …, n_{r}}_{>}, {k_{1}, …, k_{r}}_{>} ⊆ N. Then

(h_{n−k}(x_{1}, …, x_{k}))_{n,k∈N} ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = \frac{1}{x^{(k_{1},...,k_{r})′}} ∑_{T} x^{Δ(t(T))}

where the sum is over all row strict shifted plane partitions T of shape (k_{1}, …, k_{r}) and coshape (n_{1}, …, n_{r}).

Proof. It follows from (5) that

h_{n−k}(x_{1}, …, x_{k}) = [y_{1} … y_{k} s_{(n)} (x_{1}, …, x_{k}/\frac{1}{y_{1}} , …, \frac{1}{y_{k}} )]_{|y=0} ,  (26)

for all n, k ∈ N. But, by Theorem 3.3 we have that

∏_{i=1}^{r}(y_{1} … y_{k_{i}}) S(x, y^{−1})^{t} ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∏_{i=1}^{r}(y_{1} … y_{k_{i}}) ∑_{π} x^{t(π̃)−ḋ(π̃)−S(t(π))} y^{−ḋ(π̃)}
= ∑_{π} y^{d(π̃)−ḋ(π̃)} x^{t(π̃)−ḋ(π̃)−S(t(π))}  (27)

where the sum is over all shifted dotted plane partitions π of shape (k_{1}+1, …, k_{r}+1) in which the i-th row has smallest part ≐ ˙1 and largest part = n_{i}+1, for i = 1, …, r. Letting now y = 0 in (27) we see that the only π which give a non-zero contribution are those for which d_{i}(π̃) = ḋ_{i}(π̃), for i ≥ 1. These are exactly those π for which every part of π̃ (and hence of π) is dotted, and hence those π which are row strict. The result now follows from (26). □

Reasoning as in the proof of the preceding theorem, but using Theorem 3.4, we obtain the following result which, though less elegant than the equivalent Corollary 5.4, will be needed in §8, (see Theorem 8.2).

Theorem 5.6 Let {m_{1}, …, m_{r}}_{<}, {k_{1}, …, k_{r}}_{<} ⊆ N and m, k ∈ P, m > max{m_{r}, k_{r}}. Then

(h_{n−k}(x_{1}, …, x_{k}))_{n∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = \frac{1}{(x_{1} … x_{k})^{r}} ∑_{T} x^{t(T̃)−S(t(T))}

where the sum is over all row strict shifted plane partitions T of shape ((k + 1)^{r}) in which the i-th row has smallest part = m − k_{i} + 1 and largest part = m − m_{i} + 1. □

We now consider determinants of certain symmetric functions that are essentially, the Hall-Littlewood symmetric functions. Let α ∈ R and P_{λ}(x_{1}, …, x_{n}; α) denote the Hall-Littlewood symmetric function corresponding to the partition λ (we refer the reader to [18, Chap. III] for the definition and further information about Hall-Littlewood symmetric functions). Following [18, p.106] we let, for n ∈ P,

q_{0}(x_{1}, …, x_{n}; α) =^{def} 1

and

q_{k}(x_{1}, …, x_{n}; α) =^{def} (1 − α) P_{(k)} (x_{1}, …, x_{n}; α)

for k ∈ P.

Theorem 5.7 Let {n_{1}, …, n_{r}}_{>}, {k_{1}, …, k_{r}}_{>} ⊆ N. Then

(q_{k}(x_{1}, …, x_{n}; α))_{n,k∈N} ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∑_{T} x^{m(T)} (−α)^{|U(T)|} (1 − α)^{|F(T)|} , (28)

where the sum is over all diagonal strict shifted plane partitions T of shape (k_{1}, …, k_{r}) in which the i-th row has largest part ≤ n_{i} and ≥ n_{i+1} + 1, for i = 1, …, r (where

n_{r+1} =^{def} −1).

Proof. It is known (see [18, p.106, eq.(2.10)]) that

∑_{k≥0} q_{k}(x_{1}, … x_{n}; α) t^{k} = ∏_{i=1}^{n} \frac{1 − α x_{i} t}{1 − x_{i} t} .

Hence, we conclude from (6) that

q_{k}(x_{1}, …, x_{n}; α) = s_{(k)}(x_{1}, …, x_{n}/ − αx_{1}, …, −αx_{n}) .  (29)

Therefore

(q_{k}(x_{1}, …, x_{n}; α))_{n,k∈N} ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = S(x, −α x) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} )

and the thesis follows from Theorem 4.3.□

Reasoning as in the proof of the preceding theorem but using Theorem 4.4, we obtain the following result.

Theorem 5.8 Let {m_{1}, …, m_{r}}_{<}, {k_{1}, …, k_{r}}_{<} ⊆ N and m, n ∈ P, m > max{m_{r}, k_{r}}. Then

\{q_{k}(x_{1}, …, x_{n}; α)\}_{k∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = ∑_{T} x^{m(T)} (−α)^{|U(T)|} (1 − α)^{|F(T)|} (30)

where the sum is over all diagonal strict skew plane partitions T of shape (m − m_{1} + 1, …, m − m_{r} + r) \ (m − k_{1} + 1, …, m − k_{r} + r) with largest part ≤ n. □

Note that the symmetric function on the LHS of (30) is just the symmetric function S_{λ/μ}(x; α) defined (in the case that μ = ∅) by Macdonald in [18, p.116, eq.

(4.5)], where λ =^{def} (m − m_{1} + 1, …, m − m_{r} + r) and μ =^{def} (m − k_{1} + 1, …, m − k_{r} +

r). Therefore Theorem 5.8 gives a combinatorial interpretation of these symmetric functions. The symmetric functions S_{λ}(x; α) possess many interesting properties including that of being a basis of Λ[α] orthogonal to the basis of the Schur functions, with respect to the inner product defined by

⟨p_{λ}, p_{μ}⟩ =^{def} δ_{λ,μ} z_{λ} ∏_{i≥1}(1 − α^{λ_{i}})^{−1} ,  (31)

where z_{λ} = ∏_{i≥1} i^{m_{i}(λ)} m_{i}(λ)!. We refer the reader to [18, Chap. III, §4] for details.

A particularly interesting case of the preceding two theorems occurs when α = −1. In fact, it follows from [18, p.134, Ex.8], (see also [31, §6]) that

q_{k}(x_{1}, …, x_{n}; −1) = Q_{(k)}(x_{1}, …, x_{n}) ,

where Q_{λ}(x_{1}, …, x_{n}) denotes the Schur’s Q-function associated to the partition λ. These functions were originally defined by Schur [25] in connection with the study of the projective representations of the symmetric group and have since been widely studied (see,e.g., [11], [21], [31]).

6 q-Binomial Determinants

In this section we consider various determinants of q-binomial coefficients. These are defined by

[\begin{matrix}n \\ k \end{matrix}]_{q} =^{def} ∏_{i=1}^{k} \frac{(1 − q^{n−i+1})}{(1 − q^{i})} ,

(with the convention that [\begin{matrix}n \\ 0 \end{matrix}]_{q} =^{def} 1 for all n ∈ N). We let

B(q) =^{def} ([\begin{matrix}n \\ k \end{matrix}]_{q})_{n,k∈N} .

Theorem 6.1 Let {n_{1}, … n_{r}}_{>}, {k_{1}, …, k_{r}}_{>} ⊆ N. Then

B(q) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = q^{−n((k_{1}+1,...,k_{r}+1)′)} ∑_{T} q^{|\overline{T} |}

where the sum is over all row strict shifted plane partitions T of shape (k_{1}+1, …, k_{r}+ 1) and coshape (n_{1} + 1, …, n_{r} + 1), (where for a partition λ = (λ_{1}, λ_{2}, …), n(λ) =^{def} ∑_{i≥1}(i − 1) λ_{i}).

Proof. It is well known (see, e.g., [18, p.18, Ex.3]) that

[\begin{matrix}n \\ k \end{matrix}]_{q} = h_{n−k}(1, q, q^{2}, …, q^{k}) .  (32)

Hence from Theorem 5.5 we obtain that

B(q) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∑_{T} (∏_{i≥1} q^{−(i−1) d_{i}(T)}) ∏_{i≥1} q^{(i−1) (t_{i}(T)−t_{i+1}(T))}
= ∑_{T} q^{− ∑_{i≥1}(i−1) d_{i}(T)} q^{∑_{i≥1}(i−1) (t_{i}(T)−t_{i+1}(T))}

where the sum is over all row strict shifted plane partitions T of shape (k_{1}+1, …, k_{r}+ 1) and coshape (n_{1} + 1, …, n_{r} + 1), and the thesis follows. □

In the case q = 1 the preceding theorem was first proved (though stated in a slightly different way) by Gessel and Viennot [8, Corollary 11].

Theorem 6.2 Let {m_{1}, …, m_{r}}_{<}, {k_{1}, …, k_{r}}_{<} ⊆ N and m, k ∈ P, m > max{m_{r}, k_{r}}. Then

\{[\begin{matrix}n \\ k \end{matrix}]_{q}\}_{n∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = q^{− ∑_{i=1}^{r}(m_{i}+k−k_{i})} ∑_{T} q^{|T |}

where the sum is over all column strict plane partitions T of shape (m − m_{1} + 1, …, m − m_{r} + r) \ (m + k − k_{1} + 1, …, m + k − k_{r} + r) and largest part ≤ k + 1.

Proof. The result clearly holds if k_{1} < k, so assume that k_{1} ≥ k. From (32) we conclude that

\{[\begin{matrix}n \\ k \end{matrix}]_{q}\}_{n∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = \{h_{n}(1, q, …, q^{k})\}_{n∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1} − k, …, k_{r} − k \end{matrix} )

and the thesis follows from Corollary 5.4. □

We illustrate the above theorem with an example. Suppose k = 3, {m_{1}, m_{2}, m_{3}}_{<} =^{def} {1, 2, 3}, {k_{1}, k_{2}, k_{3}}_{<} =^{def} {5, 6, 7} and m = 8. Then we are looking for column strict skew plane partitions of shape (1, 1, 1) and largest part ≤ 4. Clearly, there are 4 such plane partitions, namely

4  4  4  3
3 , 3 , 2 , 2
2  1  1  1

Therefore

\frac{∑_{T}q^{|T |}}{q^{∑_{i=1}^{r}(m_{i}+k−k_{i})}} = \frac{q^{6} + q^{7} + q^{8} + q^{9}}{q^{3}} = q^{3} + q^{4} + q^{5} + q^{6} ,

and hence,

\{[\begin{matrix}n \\ 3 \end{matrix}]_{q}\}_{n∈N} ( \begin{matrix}1 & 2 & 3 \\ 5 & 6 & 7 \end{matrix} ) = det [ \begin{matrix}[\begin{matrix}4 \\ 3 \end{matrix}]q & [\begin{matrix}5 \\ 3 \end{matrix}]q & [\begin{matrix}6 \\ 3 \end{matrix}]q \\ [\begin{matrix}3 \\ 3 \end{matrix}]q & [\begin{matrix}4 \\ 3 \end{matrix}]q & [\begin{matrix}5 \\ 3 \end{matrix}]q \\ 0 & [\begin{matrix}3 \\ 3 \end{matrix}]q & [\begin{matrix}4 \\ 3 \end{matrix}]q \end{matrix} ]

= q^{3} + q^{4} + q^{5} + q^{6} .

In [33] Yahori asked for a combinatorial interpretation of the minors of the matrix

F (q) =^{def} ([\begin{matrix}n+k \\ k \end{matrix}]_{q})_{n,k∈N}

(we use the letter F since the matrix ((\begin{matrix}n+k \\ k \end{matrix}))_{n,k∈N} is sometimes called the Fermat matrix, see, e.g., [5, p.171]), and whether its principal minors form a periodic sequence. We can easily answer these questions using (32) and Corollary 5.3.

Theorem 6.3 Let {n_{1}, …, n_{r}}_{>}, {k_{1}, …, k_{r}}_{>} ⊆ N. Then

F (q) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = q^{− ∑_{i=1}^{r} n_{i}} ∑_{T} q^{|\overline{T} |}

where the sum is over all column strict shifted plane partitions T of shape (n_{1} + 1, …, n_{r} + 1) and coshape (k_{1} + 1, …, k_{r} + 1). □

In particular, we obtain the following result.

Corollary 6.4 Let n ∈ N. Then

F (q) ( \begin{matrix}0, 1, …, n \\ 0, 1, …, n \end{matrix} ) = q^{n(n+1)(2n+1)/6} .□

7 q-Stirling Determinants of the first kind

In this section we consider determinants of q-Stirling numbers of the first kind. These were introduced by Gould in [10] and have been the subject of considerable research in recent years (see, e.g., [16], [20], [22], [24]). They are defined inductively by letting

c[n, k]_{q} =^{def} c[n − 1, k − 1]_{q} + [n − 1]_{q} c[n − 1, k]_{q}

for n ∈ P, and c[0, k]_{q} =^{def} δ_{0,k} for k ∈ N, (with the convention that c[n, k]_{q} =^{def} 0 if either n < 0 or k < 0). We let

C(q) =^{def} (c[n + 1, k + 1]_{q})_{n,k∈N} .

All the results in this section are new even in the case q = 1.

Let T = (T_{i,j})_{1≤i≤r , i+μ_{i}≤j≤λ_{i}+i−1} be a row strict shifted skew plane partition. We define the row complement of T, denoted RC(T), to be the array having as i-th row the elements of [T_{i,i+μ_{i}}]\{T_{i,i+μ_{i}}, T_{i,i+μ_{i}+1}, …, T_{i,i+λ_{i−1}}}, in decreasing order, for i = 1, …, r. For example, if

T = 10 9 7 5 4 3 1 9 7 5 4 1 5 2 ,  (33)

then

RC(T) = 8 6 2 8 6 3 2 4 3 1

Theorem 7.1 Let {n_{1}, …, n_{r}}_{>} , {k_{1}, …, k_{r}}_{>} ⊆ N. Then

C(q) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∑_{T} ∏_{i∈RC(T)}[i]_{q} ,

where the sum is over all row strict shifted plane partitions T of shape (k_{1}+1, …, k_{r}+ 1) and coshape (n_{1} + 1, …, n_{r} + 1).

Proof. It is well known (see, e.g., [22]) that

C[n + 1, k + 1]_{q} = e_{n−k}([1]_{q}, [2]_{q}, …, [n]_{q}) .

Hence, by Corollary 5.1,

C(q) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∏_{i=1}^{r} (∏_{j=1}^{n_{i}}[j]_{q}) (e_{k} ([1]_{q}^{−1}, …, [n]_{q}^{−1}))_{n,k∈N} ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} )
= ∏_{i=1}^{r} ∏_{j=1}^{n_{i}}[j]_{q} ∑_{T} ∏_{i≥1}[i]_{q}^{−m_{i}(\overline{T})}

where the sum is over all row strict shifted plane partitions T of shape (k_{1}+1, …, k_{r}+ 1) and coshape (n_{1} + 1, …, n_{r} + 1), and the thesis follows. □

We illustrate the above theorem with an example. Suppose that {k_{1}, k_{2}, k_{3}}_{>} =^{def} {2, 1, 0}, and {n_{1}, n_{2}, n_{3}}_{>} =^{def} {5, 4, 3}. Then there is only one row strict shifted plane partition T of shape (3, 2, 1) and coshape (6, 5, 4), namely

6 5 4 5 4 4

Its row complement is

3 2 1
3 2 1
3 2 1

and therefore

∏_{i∈RC(T)}[i]_{q} = ([3]_{q} [2]_{q} [1]_{q})^{3}  (34)
= q^{9} + 6q^{8} + 18q^{7} + 35q^{6} + 48q^{5} + 48q^{4} + 35q^{3} + 18q^{2} + 6q + 1 .

Hence

C(q) ( \begin{matrix}3 & 4 & 5 \\ 0 & 1 & 2 \end{matrix} ) = det [ \begin{matrix}c[4, 1]_{q} & c[4, 2]_{q} & c[4, 3]_{q} \\ c[5, 1]_{q} & c[5, 2]_{q} & c[5, 3]_{q} \\ c[6, 1]_{q} & c[6, 2]_{q} & c[6, 3]_{q} \end{matrix} ]  (35)
= q^{9} + 6q^{8} + 18q^{7} + 35q^{6} + 48q^{5} + 48q^{4} + 35q^{3} + 18q^{2} + 6q + 1

Reasoning as in the proof of the preceding theorem, but using Corollary 5.2, yields the following result.

Theorem 7.2 Let {m_{1}, …, m_{r}}_{<}, {k_{1}, …, k_{r}}_{<} ⊆ N and m, n ∈ P, m > max{m_{r}, k_{r}}. Then

\{c[n + 1, k + 1]_{q}\}_{k∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = ∑_{T} ∏_{i∈RC(T)}[i]_{q}

where the sum is over all row strict skew plane partitions T of shape (m − m_{1} + 2, …, m − m_{r} + r + 1) \ (m − k_{1} + 1, …, m − k_{r} + r) and coshape ((n + 1)^{r}). □

Even though the preceding theorems are very elegant, they are not “natural” in the sense that they don’t involve permutations, as the usual combinatorial interpretation of the q-Stirling numbers of the first kind does (see, e.g., [16]). To obtain such a “natural” combinatorial interpretation a little more work is required.

Given a permutation σ ∈ S_{n} having k cycles C_{1}, …, C_{k} we let S(σ) =^{def} {min (C_{1}), …, min (C_{k}), n + 1}, and {σ^{(1)}, …, σ^{(k+1)}}_{>} =^{def} S(σ). We say that σ is written in normal form if:

i) each cycle of σ is written with its smallest element first;

ii) the cycles are written in increasing order of their first elements.

The normal representation of σ is the word obtained from the normal form of σ by erasing all the parentheses. The number of inversions of σ, denoted by inv(σ), is the number of inversions in the normal representation of σ. More precisely,

inv(σ) =^{def} |{(i, j) ∈ [n] × [n] : i > j , a_{i} < a_{j}}| ,

where a_{1} … a_{n} is the normal representation of σ. For example, if σ = (23)(1)(87)(9465) then S(σ) = {2, 1, 7, 4, 10}, σ^{(5)} = 1, σ^{(4)} = 2, σ^{(3)} = 4, σ^{(2)} = 7, σ^{(1)} = 10, the normal form of σ is (1)(23)(4659)(78), its normal representation is 123465978, and inv(σ) = 3.

Lemma 7.3 Let n ∈ P and S ⊆ [n + 1], 1, n + 1 ∈ S. Then

∑_{{σ∈S_{n}: S(σ)=S}} q^{inv (σ)} = ∏_{a∈[n]\S}[a − 1]_{q} .  (36)

Proof. We will prove (36) by constructing an appropriate bijection. Let σ ∈ S_{n} be such that S(σ) = S and let a_{1} … a_{n} be the normal representation of σ. We define a function f_{σ} : [n] \ S → N by letting, for a ∈ [n] \ S,

f_{σ}(a) =^{def} |{i ∈ [n] : i > j , a_{i} < a}| ,  (37)

where j ∈ [n] is such that a_{j} = a. Note that, since a ∈ [n] \ S, and a_{1} … a_{n} is the normal representation of σ, f_{σ}(a) ≤ a − 2. Conversely, let f : [n] \ S → N be such that f(a) ≤ a − 2 for all a ∈ [n] \ S, and let {b_{1}, …, b_{k}}_{<} =^{def} [n] \ S. We construct a permutation σ_{f} inductively as follows. We first place b_{1} in σ^{(1)} … σ^{(n−k)} so that there are exactly f(b_{1}) elements to the right of b_{1} that are smaller than b_{1}. Note that this is always possible because f(b_{1}) ≤ b_{1} − 2 and {1, 2, …, b_{1} − 1} ⊆ {σ^{(1)}, …, σ^{(n−k)}}. Suppose now that we have already placed b_{1}, …, b_{j−1} in σ^{(1)}, …, σ^{(n−k)}. We then place b_{j} in the leftmost position such that

{1, 2, …, b_{j} − 1} ⊆ S ∪ {b_{1}, b_{2}, …, b_{j−1}} .

We then let σ_{f} be the unique permutation with S(σ_{f}) = S that has as normal representation the word w obtained at the end of the preceding process (i.e., after the insertion of b_{k}). Note that σ_{f} always exists because, for each a ∈ w, the rightmost element of S that is to the left of a is < a. In fact, by our definition, when b_{j} is inserted the element to the left of it is < b_{j} and this, by induction on j, proves our claim.

It is easy to see that the maps σ ↦ f_{σ} and f ↦ σ_{f} are inverses of each other. Furthermore, it follows from (37) that

∑_{a∈[n]\S} f_{σ}(a) = |{(i, j) ∈ [n] × [n] : i > j , a_{i} < a_{j}, a_{j} ∈ [n] \ S}|
= inv (σ) ,

since a_{1} … a_{n} is the normal representation of σ. Hence

∑_{{σ∈S_{n}: S(σ)=S}} q^{inv (σ)} = ∑_{f} ∏_{a∈[n]\S} q^{f(a)}
= ∏_{a∈[n]\S}(1 + q + q^{2} + … + q^{a−2})

(where f runs over all functions f : [n] \ S → N such that f(a) ≤ a − 2 for all a ∈ [n] \ S), as desired. □

We illustrate the preceding bijection with an example. Let n = 9 and S = {1, 2, 5, 7, 10}. If σ = (14)(23)(56)(798) then f_{σ}(3) = 0, f_{σ}(4) = 2, f_{σ}(6) = 0, f_{σ}(8) = 0, and f_{σ}(9) = 1. Conversely, if f(3) = 1, f(4) = 2, f(6) = 0, f(8) = 6, and f(9) = 4 then successive insertions of 3, 4, 6, 8, and 9 into σ^{(1)} … σ^{(4)} = 1257 yields, respectively,

13257, 143257, 1432567, 18432567, 184392567

and hence σ = (18439)(2)(56)(7).

Given an r-tuple of permutations (σ_{1}, …, σ_{r}) and a partition μ = (μ_{1}, …, μ_{r}) we associate to them a shifted skew tabloid, denoted ST_{μ}(σ_{1}, …, σ_{r}), by letting σ_{i}^{(j)} be its (i, i + μ_{i} + j − 1) entry, for i = 1, …, r, j = 1, …, k_{i} + 1 (where k_{i} is the number of cycles of σ_{i}, for i = 1, …, r). For example, if μ = (2, 1, 1), σ_{1} = (23)(1)(564), σ_{2} = (31)(2), and σ_{3} = (12)(78)(635)(4), then

ST_{μ}(σ_{1}, σ_{2}, σ_{3}) = 7 4 2 1 4 2 1 9 7 4 3 1

We will usually omit the index μ when there is no danger of confusion.

Lemma 7.4 Let T be a row strict shifted skew plane partition of coshape (n_{1} + 1, …, n_{r} + 1). Then

∏_{i∈RC(T)}[i]_{q} = ∑_{(σ_{1},...,σ_{r})} ∏_{i=1}^{r} q^{inv (σ_{i})}

where the sum is over all r-tuples (σ_{1}, …, σ_{r}) ∈ S_{n_{1}+1} × … × S_{n_{r}+1} such that ST (σ_{1}, …, σ_{r}) = T̂.

Proof. Let T =^{def} (T_{i,j})_{1≤i≤r , c_{i}+i≤j≤d_{i}+i−1} be a row strict shifted skew plane partition of coshape (n_{1} + 1, …, n_{r} + 1). Then, by the definition of RC(T), we have that

RC(T)_{i} + 1 = ([n_{i}]) \ (\overline{T})_{i}) + 1
= [n_{i} + 1] \ {T_{i,c_{i}+i+1} + 1, …, T_{i,d_{i}+i−1} + 1, 1} ,

for i = 1, …, r. Hence, by Lemma 7.3, we have that

∏_{i∈RC(T)}[i]_{q} = ∏_{i=1}^{r} ∏_{j∈RC(T)_{i}}[j]_{q}
= ∏_{i=1}^{r} ∏_{j∈RC(T)_{i}+1}[j − 1]_{q}
= ∏_{i=1}^{r} ∑_{σ} q^{inv (σ)} ,

where the sum is over all σ ∈ S_{n_{i}+1} such that S(σ) = {n_{i}+2, T_{i,c_{i}+i+1}+1, …, T_{i,d_{i}+i−1}+ 1, 1}, and the result follows. □

Since a shifted skew tabloid of the form ST (σ_{1}, …, σ_{r}) with (σ_{1}, …, σ_{r}) ∈ S_{n_{1}+1} × … × S_{n_{r}+1} is necessarily row strict and of coshape (n_{1} + 2, …, n_{r} + 2) from the preceding lemma and Theorems 7.1 and 7.2 we deduce the following results.

Theorem 7.5 Let {n_{1}, …, n_{r}}_{>}, {k_{1}, …, k_{r}}_{>} ⊆ N. Then

C(q) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∑_{(σ_{1},...,σ_{r})} ∏_{i=1}^{r} q^{inv (σ_{i})}

where the sum is over all r-tuples (σ_{1}, …, σ_{r}) ∈ S_{n_{1}+1} × … × S_{n_{r}+1} such that ST (σ_{1}, …, σ_{r}) is a shifted plane partition of shape (k_{1} +2, …, k_{r} +2), (equivalently, such that σ_{i} has k_{i} + 1 cycles and σ_{i}^{(j)} ≥ σ_{i+1}^{(j−1)}, for 1 ≤ i ≤ r − 1 and 2 ≤ j ≤ k_{i} + 2). □.

Theorem 7.6 Let {m_{1}, …, m_{r}}_{<}, {k_{1}, …, k_{r}}_{<} ⊆ N and m, n ∈ P, m > max{m_{r}, k_{r}}. Then

\{c[n + 1, k + 1]_{q}\}_{k∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = ∑_{(σ_{1},...,σ_{r})} ∏_{i=1}^{r} q^{inv (σ_{i})}

where the sum is over all r-tuples (σ_{1}, …, σ_{r}) ∈ (S_{n+1})^{r} such that ST (σ_{1}, …, σ_{r}) is a skew plane partition of shape (m−m_{1}+3, …, m−m_{r} +r+2)\(m−k_{1}+1, …, m− k_{r} + r), (equivalently, such that σ_{i} has k_{i} − m_{i} +1 cycles and σ_{i}^{(k_{i}−j)} ≥ σ_{i+1}^{(k_{i+1}−j−1)} for 1 ≤ i ≤ r − 1 and m_{i+1} − 3 ≤ j ≤ k_{i} − 1). □

We illustrate Theorem 7.5, when q = 1, with an example. Suppose that {k_{1}, k_{2}, k_{3}}_{>} =^{def} {2, 1, 0}, and {n_{1}, n_{2}, n_{3}}_{>} =^{def} {5, 4, 3}, as in the example following Theorem 7.1. Then we are looking for triples (σ_{1}, σ_{2}, σ_{3}) ∈ S_{6} × S_{5} × S_{4} such that ST (σ_{1}, σ_{2}, σ_{3}) is a shifted plane partition of shape (4, 3, 2). Since (σ_{1}, σ_{2}, σ_{3}) ∈ S_{6} × S_{5} × S_{4}, ST (σ_{1}, σ_{2}, σ_{3}) must be row strict and of the form

\begin{aligned}7 & · & · & 1 \\ & 6 & · & 1 \\ & & 5 & 1 \end{aligned}

Hence the only possibility for ST (σ_{1}, σ_{2}, σ_{3}) is

7 6 5 1 6 5 1 5 1 This means that we are looking for triples (σ_{1}, σ_{2}, σ_{3}) ∈ S_{6} × S_{5} × S_{4} such that S(σ_{1}) = {7, 6, 5, 1}, S(σ_{2}) = {6, 5, 1}, and S(σ_{3}) = {5, 1}. Hence σ_{1}, σ_{2}, and σ_{3} must be of the form

σ_{1} = (1 x_{1} x_{2} x_{3})(5)(6) ,
σ_{2} = (1 y_{1} y_{2} y_{3})(5) ,
σ_{3} = (1 z_{1} z_{2} z_{3}) ,

where x_{1} x_{2} x_{3}, y_{1} y_{2} y_{3}, and z_{1} z_{2} z_{3} are any permutations of {2, 3, 4}. Hence there are 6 possibilities for each one of σ_{1}, σ_{2}, and σ_{3} and therefore 216 possible triples (σ_{1}, σ_{2}, σ_{3}), which is in accordance with (34) and (35) when q = 1.

8 q-Stirling Determinants of the second kind

In this section we consider determinants of q-Stirling numbers of the second kind. These were introduced by Gould in [10] and have been the subject of considerable interest recently (see, e.g, [16], [20], [23], [32]). They are defined inductively by letting

S[n, k]_{q} = S[n − 1, k − 1]_{q} + [k]_{q} S[n − 1, k]_{q}

for n ∈ P, and S[0, k]_{q} =^{def} δ_{0,k} for k ∈ N, (with the convention that S[n, k]_{q} =^{def} 0 if either n < 0 or k < 0). We let

S(q) =^{def} (S[n + 1, k + 1]_{q})_{n,k∈N} .

All the results in this section are new even in the case q = 1.

Theorem 8.1 Let {n_{1}, …, n_{r}}_{>}, {k_{1}, …, k_{r}}_{>} ⊆ N. Then

S(q) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∑_{T} ∏_{i≥1}[i]_{q}^{t_{i}(T)−t_{i+1}(T)−d_{i}(T)}

where the sum is over all row strict shifted plane partitions T of shape (k_{1}+1, …, k_{r}+ 1) and coshape (n_{1} + 1, …, n_{r} + 1).

Proof. It is well known (see, e.g., [22]) that

S[n + 1, k + 1]_{q} = h_{n−k}([1]_{q}, [2]_{q}, …, [k + 1]_{q}) .

Hence, by Theorem 5.5,

S(q) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∑_{T} ∏_{i≥1}[i]_{q}^{t_{i}(T)−t_{i+1}(T)−d_{i}(T)}

(since (k_{1} + 1, …, k_{r} + 1)′ = (d_{1}(T), d_{2}(T), …)) where the sum is over all row strict shifted plane partitions T of shape (k_{1}+1, … k_{r} +1) and coshape (n_{1}+1, …, n_{r} +1), as desired. □

We illustrate the above theorem with an example. Suppose that {k_{1}, k_{2}, k_{3}}_{>} =^{def} {2, 1, 0}, and {n_{1}, n_{2}, n_{3}}_{>} =^{def} {6, 5, 4}. Then there is only one row strict shifted plane partition T of shape (3, 2, 1) and coshape (7, 6, 5), namely,

7 6 5 6 5 5 .

Therefore

t(T) = (18, 11, 5, 0, …) ,
Δ(t(T)) = (7, 6, 5, 0, …) ,
d(T) = (3, 2, 1, 0, …) ,

hence

∏_{i≥1}[i]_{q}^{t_{i}(T)−t_{i+1}(T)−d_{i}(T)} = [1]_{q}^{4} [2]_{q}^{4} [3]_{q}^{4} ,  (38)

and

S(q) ( \begin{matrix}4 & 5 & 6 \\ 0 & 1 & 2 \end{matrix} ) = det [ \begin{matrix}S[5, 1]_{q} & S[5, 2]_{q} & S[5, 3]_{q} \\ S[6, 1]_{q} & S[6, 2]_{q} & S[6, 3]_{q} \\ S[7, 1]_{q} & S[7, 2]_{q} & S[7, 3]_{q} \end{matrix} ]
= ([1]_{q} [2]_{q} [3]_{q})^{4}  (39)
= 1 + 8q + 32q^{2} + 84q^{3} + 160q^{4} + 232q^{5} + 262q^{6} + 232q^{7} + 160q^{8}
+ 84q^{9} + 32q^{10} + 8q^{11} + q^{12} .

Reasoning as in the proof of the last theorem, but using Theorem 5.6, gives the following result.

Theorem 8.2 Let {m_{1}, …, m_{r}}_{<}, {k_{1}, …, k_{r}}_{<} ⊆ N and m, k ∈ P, m > max{m_{r}, k_{r}}. Then

\{S[n + 1, k + 1]_{q}\}_{n∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = ∑_{π} ∏_{i=1}^{k+1}[i]_{q}^{t_{i}(π)−t_{i+1}(π)−r}

where the sum is over all row strict shifted plane partitions π of shape ((k + 2)^{r}) in which the i-th row has largest part = m − m_{i} + 1 and smallest part = m − k_{i}, for i = 1, …, r. □

Note that it is possible to use Corollary 5.4 instead of Theorem 5.6 to obtain an equivalent version of the preceding theorem. However the version that we have chosen is much better suited for the applications that will follow (see Lemmas 8.4 and 8.5).

We illustrate Theorem 8.2 with an example. Suppose that k = 2, {m_{1}, m_{2}, m_{3}}_{<} =^{def} {1, 2, 3}, {k_{1}, k_{2}, k_{3}}_{<} =^{def} {5, 6, 7}, and m = 8. Then we are looking for row strict shifted plane partitions of the form

\begin{aligned}8 & · & · & 3 \\ & 7 & · & · & 2 \\ & & 6 & · & · & 1 \end{aligned}

Clearly, there is only one such row strict shifted plane partition, namely

T = 8 7 6 3 7 6 3 2 6 3 2 1

Therefore

∏_{i=1}^{3}[i]_{q}^{t_{i}(T)−t_{i+1}(T)−3} = [1]_{q}^{2} [2]_{q}^{2} [3]_{q}^{2} ,  (40)

and hence

\{S[n + 1, 3]_{q}\}_{n∈N} ( \begin{matrix}1 & 2 & 3 \\ 5 & 6 & 7 \end{matrix} ) = det [ \begin{matrix}S[5, 3]_{q} & S[6, 3]_{q} & S[7, 3]_{q} \\ S[4, 3]_{q} & S[5, 3]_{q} & S[6, 3]_{q} \\ S[3, 3]_{q} & S[4, 3]_{q} & S[5, 3]_{q} \end{matrix} ]
= ([1]_{q} [2]_{q} [3]_{q})^{2}
= 1 + 4q + 8q^{2} + 10q^{3} + 8q^{4} + 4q^{5} + q^{6} .

The same remark made after Theorem 7.2 applies to the last two results. In order to obtain “natural” versions of these results we will need analogs of Lemmas 7.3 and 7.4 for (set) partitions.

Let m, n ∈ P, m < n. Given a partition π = {B_{1}, …, B_{k}} of [m, n] into k blocks

we let S(π) =^{def} {max (B_{1}), …, max (B_{k}), m−1} and {π^{(1)}, …, π^{(k+1)}}_{>} =^{def} S(π). Let

now π_{i} be the (unique) block of π containing π^{(i)}, for i = 1, …, k. We define the height of π to be the number

ht (π) =^{def} ∑_{i=1}^{k}(i − 1) (|π_{i}| − 1) .

For example, if π = 48/569/7 then S(π) = {8, 9, 7, 3}, π^{(1)} = 9, π^{(2)} = 8, π^{(3)} = 7, π^{(4)} = 3, π_{1} = {5, 6, 9}, π_{2} = {4, 8}, π_{3} = {7}, and

ht (π) = 1 · 1 + 2 · 0 = 1 .

The verification of the next technical result is easy and is left to the reader.

Lemma 8.3 Let n ∈ P and S =^{def} {s_{1}, …, s_{k+1}}_{>} ⊆ [n] ∪ {0}, 0, n ∈ S. Then

∑_{{π∈Π([n]) : S(π)=S}} q^{ht (π)} = [k]_{q}^{s_{k}−1} ∑_{{π∈Π([s_{k}+1,n]) : S(π)=S\{0}}} q^{ht (π)} .□

Lemma 8.4 Let n ∈ P and S =^{def} {s_{1}, …, s_{k+1}}_{>} ⊆ [n] ∪ {0}, 0, n ∈ S. Then

∑_{{π∈Π([n]): S(π)=S}} q^{ht(π)} = ∏_{i=1}^{k}[i]_{q}^{s_{i}−s_{i+1}−1} .  (41)

Proof. We will find it convenient to introduce the following notation , for a ∈ [n]\S we let i(a) be the unique element of [k] such that s_{i(a)+1} < a < s_{i(a)}.

We will prove (41) by constructing an appropriate bijection. Let π be a partition of [n] such that S(π) = S. For a ∈ [n] \ S we let

f_{π}(a) =^{def} j − 1  (42)

if a and s_{j} are in the same block of π. Note that since S(π) = S, f_{π}(a) ≤ i(a) − 1, for all a ∈ [n] \ S. Conversely, given a function f : [n] \ S → N such that f(a) ≤ i(a) − 1 for all a ∈ [n] \ S, we construct a partition π_{f} of [n] by putting a ∈ [n] \ S in the same block as s_{f(a)+1} and putting s_{1}, …, s_{k} all in different blocks. Note that a < s_{i(a)} ≤ s_{f(a)+1} so that S(π_{f}) = S. It is easy to see that the maps π ↦ f_{π} and f ↦ π_{f} are inverses of each other. Furthermore, it follows from (42) that

ht(π) = ∑_{a∈[n]\S} f_{π}(a) .

Hence

∑_{{π∈Π([n]) : S(π)=S}} q^{ht(π)} = ∑_{f} ∏_{a∈[n]\S} q^{f(a)}
= ∏_{a∈[n]\S}(1 + q + q^{2} + … + q^{i(a)−1})
= ∏_{i=1}^{k}(1 + q + q^{2} + … + q^{i−1})^{s_{i}−s_{i+1}−1} ,

(where f runs over all functions f : [n] \ S → N such that f(a) ≤ i(a) − 1 for all a ∈ [n] \ S), and the thesis follows. □

We illustrate the above construction with an example. Let n = 9 and S =^{def} {9, 7, 6, 2}, so that s_{1} = 9, s_{2} = 7, s_{3} = 6, s_{4} = 2 and hence i(1) = 4, i(3) = i(4) = i(5) = 3, and i(8) = 1. If π = 938/75/64/21 then f_{π}(3) = f_{π}(8) = 0, f_{π}(5) = 1, f_{π}(4) = 2, and f_{π}(1) = 3. Conversely, if f(1) = 3, f(3) = 2, f(4) = f(5) = 1, and f(8) = 0 then π_{f} = 98/745/63/21.

Given an r-tuple of partitions (π_{1}, …, π_{r}) we associate to it a shifted tabloid ST (π_{1}, … π_{r}) by letting the elements of S(π_{i}) (in decreasing order) be the i-th row of it, for i = 1, …, r, and then shifting the resulting array. For example, if π_{1} = 82/7591/6/354, π_{2} = 25/134, and π_{3} = 71/52/3/46 then

ST (π_{1}, π_{2}, π_{3}) = 9 8 6 5 5 4 7 6 5 3

Lemma 8.5 Let T be a row strict shifted plane partition of coshape (n_{1}+1, …, n_{r} + 1). Then

∏_{j≥1}[j]_{q}^{t_{j}(T)−t_{j+1}(T)−d_{j}(T)} = ∑_{(π_{1},...,π_{r})} ∏_{i=1}^{r} q^{ht (π_{i})}  (43)

where the sum is over all r-tuples (π_{1}, …, π_{r}) ∈ Π([n_{1} + 1]) × … × Π([n_{r} + 1]) such that ST (π_{1}, …, π_{r}) = T.

Proof. Let T = (T_{i,j})_{1≤i≤r , i≤j≤k_{i}+i} be a row strict shifted plane partition satisfying the hypotheses of the lemma. Then, for 1 ≤ j ≤ k_{1} + 1,

t_{j}(T) − t_{j+1}(T) − d_{j}(T) = ∑_{i=1}^{d_{j}(T)}(T_{i,i+j−1} − T_{i,i+j} − 1) ,

hence, by Lemma 8.4,

∏_{j=1}^{k_{1}+1}[j]_{q}^{t_{j}(T)−t_{j+1}(T)−d_{j}(T)} = ∏_{j=1}^{k_{1}+1} ∏_{i=1}^{d_{j}(T)}[j]_{q}^{T_{i,i+j−1}−T_{i,i+j}−1}
= ∏_{i=1}^{r} ∏_{j=1}^{k_{i}+1}[j]_{q}^{T_{i,i+j−1}−T_{i,i+j}−1}
= ∏_{i=1}^{r} ∑_{{π∈Π([n_{i}+1]) : S(π)=T_{i}∪{0}}} q^{ht (π)} ,

and (43) follows. □

Since a shifted tabloid of the form ST (π_{1}, …, π_{r}) with (π_{1}, …, π_{r}) ∈ Π([n_{1} + 1])×…×Π([n_{r} +1]) is necessarily row strict and of coshape (n_{1} +1, …, n_{r} +1) from Theorems 8.1 and 8.2 and the preceding lemma we deduce the following results.

Theorem 8.6 Let {n_{1}, …, n_{r}}_{>}, {k_{1}, …, k_{r}}_{>} ⊆ N. Then

S(q) ( \begin{matrix}n_{r}, …, n_{1} \\ k_{r}, …, k_{1} \end{matrix} ) = ∑_{(π_{1},...,π_{r})} ∏_{i=1}^{r} q^{ht(π_{i})} ,

where the sum is over all r-tuples (π_{1}, …, π_{r}) ∈ Π([n_{1} + 1]) × … × Π([n_{r} + 1]) such that ST (π_{1}, …, π_{r}) is a shifted plane partition of shape (k_{1} + 1, …, k_{r} + 1), (equivalently, such that π_{i} has k_{i} + 1 blocks and π_{i}^{(j)} ≥ π_{i+1}^{(j−1)}, for 1 ≤ i ≤ r − 1 and 2 ≤ j ≤ k_{i} + 1). □

Theorem 8.7 Let {m_{1}, …, m_{r}}_{<}, {k_{1}, …, k_{r}}_{<} ⊆ N and m, k ∈ P, m > max{m_{r}, k_{r}}. Then

\{S[n + 1, k + 1]_{q}\}_{n∈N} ( \begin{matrix}m_{1}, …, m_{r} \\ k_{1}, …, k_{r} \end{matrix} ) = ∑_{(π_{1},...,π_{r})} ∏_{i=1}^{r} q^{ht (π_{i})}

where the sum is over all r-tuples (π_{1}, …, π_{r}) ∈ Π([m − k_{1} + 1, m − m_{1} + 1]) × … × Π([m − k_{r} + 1, m − m_{r} + 1]) such that ST (π_{1}, …, π_{r}) is a shifted plane partition of shape ((k + 2)^{r}), (equivalently, such that π_{i} has k + 1 blocks, for i = 1, …, r, and π_{i}^{(j)} ≥ π_{i+1}^{(j−1)} for 1 ≤ i ≤ r − 1 and 2 ≤ j ≤ k + 2).

Proof. Let T be a row strict shifted plane partition of shape ((k + 2)^{r}) in which the i-th row has largest part = m − m_{i} + 1 and smallest part = m − k_{i}, for i = 1, …, r. Then by Lemmas 8.3 and 8.5 we obtain that

∏_{j≥1}[j]_{q}^{t_{j}(T)−t_{j+1}(T)−d_{j}(T)} = ∏_{i=1}^{r} ∑_{{π∈Π([m−m_{i}+1]): S(π)=T_{i}∪{0}}} q^{ht (π)}
= ∏_{i=1}^{r}[k + 2]_{q}^{m−k_{i}−1} ∑_{{π∈Π([m−k_{i}+1,m−m_{i}+1]): S(π)=T_{i}}} q^{ht (π)}
= [k + 2]^{∑_{i=1}^{r}(m−k_{i}−1)} ∑_{(π_{1},...,π_{r})} ∏_{i=1}^{r} q^{ht (π_{i})}

where the sum is over all r-tuples (π_{1}, …, π_{r}) ∈ Π([m − k_{1} + 1, m − m_{1} + 1]) × … × Π([m − k_{r} + 1, m − m_{r} + 1]) such that ST (π_{1}, …, π_{r}) = T, and the thesis follows from Theorem 8.2.□

We illustrate Theorem 8.6, when q = 1, with an example. Suppose that {k_{1}, k_{2}, k_{3}}_{>} =^{def} {2, 1, 0}, and {n_{1}, n_{2}, n_{3}}_{>} =^{def} {6, 5, 4}, as in the example following Theorem 8.1.

Then we are looking for triples (π_{1}, π_{2}, π_{3}) ∈ Π([7]) × Π([6]) × Π([5]) such that ST (π_{1}, π_{2}, π_{3}) is a shifted plane partition of shape (3, 2, 1). Since (π_{1}, π_{2}, π_{3}) ∈ Π([7]) × Π([6]) × Π([5]), ST (π_{1}, π_{2}, π_{3}) must be row strict and of the form

\begin{aligned}7 & · & · \\ & 6 & · \\ & & 5 \end{aligned}

Hence the only possibility for ST (π_{1}, π_{2}, π_{3}) is

7 6 5 6 5 5

This means that we are looking for triples (π_{1}, π_{2}, π_{3}) ∈ Π([7]) × Π([6]) × Π([5]) such that S(π_{1}) = {7, 6, 5}, S(π_{2}) = {6, 5}, and S(π_{3}) = {5}. Hence there are 3^{4} possibilities for π_{1}, 2^{4} possibilities for π_{2}, and one possibility for π_{3}. Therefore there are 1296 possible triples (π_{1}, π_{2}, π_{3}), which is in accordance with (38) and (39) when q = 1.

References

[1] A. B. Balantekin and I. Bars, Representations of supergroups, J. Math. Phys., 22 (1981), 1810-1818.

[2] A. Berele and A. Regev, Hook Young diagrams with applications to combinatorics and to representations of Lie superalgebras, Adv. in Math., 64 (1987), 118-175.

[3] F. Brenti, Combinatorial methods in the theory of Total Positivity, in preparation.

[4] L. M. Butler, The q-log concavity of q-binomial coefficients, J. Combin. Theory Ser. A, 54 (1990), 54-63.

[5] L. Comtet, Advanced Combinatorics, Reidel, Dordrecht/Boston, 1974.

[6] P. H. Dondi and P. D. Jarvis, Diagram and superfield techniques in the classical superalgebras, J. Phys., A 14 (1981), 547-563.

[7] I. M. Gelfand and M. L. Tseitlin, Finite-dimensional representations of the group of unimodular matrices (Russian), Dokl. Akad. Nauk. USSR, 71 (1950), 825-828.

[8] I. Gessel and G. Viennot, Binomial determinants, paths, and hook length formulae, Adv. in Math., 58 (1985), 300-321.

[9] I. Gessel and G. Viennot, Determinants, paths, and plane partitions, preprint.

[10] H. W. Gould, The q-Stirling numbers of the first and second kinds, Duke Math. J., 28 (1961), 281-289.

[11] T. Józefiak and P. Pragacz, A determinantal formula for skew Schur Qfunctions, preprint.

[12] V. G. Kac̆, Lie superalgebras, Adv. in Math., 26 (1977), 8-96.

[13] V. G. Kac̆, Representations of classical Lie superalgebras, in Differential Geometry Methods in Mathematical Physics II (K. Bleuler, H. R. Petry, and A. Reetz, Eds.), Lecture Notes in Math., No. 676, Springer, New York, 1978, pp. 597-626.

[14] S. Karlin, Total Positivity, vol.1, Stanford University Press, 1968.

[15] C. Krattenthaler, On the q-log-concavity of Gaussian binomial coefficients, preprint.

[16] P. Leroux, Reduced matrices and q-log concavity properties of q-Stirling numbers, J. Combin. Theory Ser. A, 54 (1990), 64-84.

[17] B. Lindström, On the vector representations of induced matroids, Bull. London Math. Soc., 5 (1973), 85-90.

[18] I. G. Macdonald, Symmetric Functions and Hall Polynomials, Oxford University Press, New York/London, 1979.

[19] N. Metropolis, G. Nicoletti, and G. C. Rota, A new class of symmetric functions, Mathematical Analysis and Applications, Part B, Adv. in Math. Suppl. Stud., 7B (1981), 563-575.

[20] S. C. Milne, Restricted growth functions, rank row matchings of partition lattices, and q-Stirling numbers, Adv. in Math., 43 (1982), 173-196.

[21] B. E. Sagan, Shifted tableaux, Schur Q-functions and a conjecture of R. Stanley, J. Combin. Theory Ser. A, 45 (1987), 62-103.

[22] B. E. Sagan, Log concave sequences of symmetric functions and analogs of the Yacobi-Trudi determinants, to appear in Trans. Amer. Math. Soc.

[23] B. E. Sagan, A maj statistic for set partitions, preprint.

[24] B. E. Sagan, Inductive proofs of q-log concavity, preprint.

[25] I. Schur, Über die Darstellung der symmetrischen und der alternierenden Gruppe durch gebrochene lineare Substitutionen, J. Reine Angew. Math., 139 (1911), 155-250.

[26] R. Stanley, Theory and application of plane partitions, Parts 1, 2, Stud. Appl. Math., 50 (1971), 167-188, 259-279.

[27] R. Stanley, Unimodality and Lie superalgebras, Studies in Applied Math., 72 (1985), 263-281.

[28] R. Stanley, Enumerative Combinatorics , vol.1, Wadsworth and Brooks/Cole, Monterey, CA, 1986.

[29] R. Stanley, A baker’s dozen of conjectures concerning plane partitions, Combinatoire Enumerative, (G. Labelle and P. Leroux Eds.), Lecture Notes in Mathematics, no.1234, Springer-Verlag, Berlin-New York, 1986, 285-293.

[30] J. R. Stembridge, Shifted tableaux and the projective representations of symmetric groups, Adv. in Math., 74 (1989), 87-134.

[31] J. R. Stembridge, Non-intersecting paths, pfaffians and plane partitions, Adv. in Math., 83 (1990), 96-131.

[32] M. Wachs and D. White, p,q-Stirling numbers and set partition statistics, preprint.

[33] Yahory, Problem Session of the Conference in Combinatorics and Commutative Algebra, Nagoya, 1990.