Simultaneous preference estimation and heterogeneity control for choice-based conjoint via support vector machines

Abstract

Support vector machines (SVMs) have been successfully used to identify individuals’ preferences in conjoint analysis. One of the challenges of using SVMs in this context is to properly control for preference heterogeneity among individuals to construct robust partworths. In this work, we present a new technique that obtains all individual utility functions simultaneously in a single optimization problem based on three objectives: complexity reduction, model fit, and heterogeneity control. While complexity reduction and model fit are dealt using SVMs, heterogeneity is controlled by shrinking the individual-level partworths toward a population mean. The proposed approach is further extended to kernel-based machines, conferring flexibility to the model by allowing nonlinear utility functions. Experiments on simulated and real-world datasets show that the proposed approach in its linear form outperforms existing methods for choice-based conjoint analysis.

Appendices

Appendix A

Strictly convexity of problem (6)

In order to prove that our Formulation (6) is strictly convex, we first rewrite it in a compact form. For this, we follow the derivation of Yajima (2005) for multicategory SVM. Let us denote by

$$\begin{aligned} \widetilde{\mathbf{w}}=[{\mathbf{w}}_0^\top ,\mathbf{w}_1^\top ,\ldots ,\mathbf{w}_N^\top ]^\top \in {\mathfrak {R}}^{J(N+1)}, \end{aligned}$$

and

$$\begin{aligned} {\mathcal{Q}}(\theta )=\left[ \begin{array}{lllll} N\theta I_J &{} -\theta I_J &{}-\theta I_J &{}\cdots &{} -\theta I_J \\ -\theta I_J &{} (1+\theta )I_J &{} 0 &{} \cdots &{} 0 \\ -\theta I_J &{}0 &{}\ddots &{}\ddots &{}\vdots \\ \vdots &{} \vdots &{} \ddots &{} \ddots &{} 0 \\ -\theta I_J &{} 0 &{} \cdots &{} 0 &{} (1+\theta )I_J \end{array} \right] \in {\mathfrak {R}}^{J(N+1)\times J(N+1)}, \end{aligned}$$

(11)

where $I_J$ denotes the identity matrix of size J. Then, the quadratic term in (6) can be expressed as

$$\begin{aligned} \sum _{i=1}^N(\left\| \mathbf{w}_i\right\| ^2 +\theta \left\| \mathbf{w}_i-\mathbf{w}_0\right\| ^2) = \tilde{\mathbf{w}}^\top {\mathcal{Q}}(\theta ) \tilde{\mathbf{w}}. \end{aligned}$$

(12)

Proposition 1

For any $\theta >0$, the matrix ${\mathcal{Q}}(\theta )$ is symmetric definite positive. Moreover,

$$\begin{aligned} {\mathcal{Q}}(\theta )^{-1}=\frac{1}{N}\left[ \begin{array}{lllll} \frac{\theta +1}{\theta } I_J &{} I_J &{} I_J &{}\cdots &{} I_J \\ I_J &{} \frac{\theta +N}{\theta +1} I_J &{} \frac{\theta }{\theta +1}I_J &{} \cdots &{} \frac{\theta }{\theta +1}I_J \\ I_J &{}\frac{\theta }{\theta +1}I_J &{}\ddots &{}\ddots &{}\vdots \\ \vdots &{} \vdots &{} \ddots &{} \ddots &{} \frac{\theta }{(\theta +1)}I_J \\ I_J &{} \frac{\theta }{\theta +1}I_J &{} \cdots &{} \frac{\theta }{(\theta +1)}I_J &{} \frac{\theta +N}{\theta +1}I_J \end{array} \right] . \end{aligned}$$

(13)

Proof

It is clear that the matrix ${\mathcal{Q}}(\theta )$ is symmetric definite positive (cf. (12)). Now, we denote by $F_i\in {\mathfrak {R}}^{J\times J(N+1)}$ and $C_i\in {\mathfrak {R}}^{J(N+1)\times J}$ the i-th block (in row) and the i-th block (in column) of ${\mathcal{Q}}(\theta )$ and ${\mathcal{Q}}(\theta )^{-1}$, respectively. Then,

$$\begin{aligned} F_1C_1=I_J,\quad F_1C_i=\frac{1}{N}\left(N\theta -\frac{N\theta +\theta ^2N}{\theta +1}\right)=0,\ i=2,\ldots ,N+1, \end{aligned}$$

$$\begin{aligned} F_iC_1=0,\quad F_iC_i=I_J,\quad F_iC_j=\frac{1}{N}(\theta -\theta )=0, i\ne j. \end{aligned}$$

Thus, the result follows. $\square$

Appendix B

Dual formulation of problem (6)

Let us denote by ${\varvec{\xi }}_t^k=(\xi _{1t}^k,\ldots ,\xi _{Nt}^k)\in {\mathfrak {R}}^N$, and by $\mathbf{X}^k_{t}=\left[ \begin{array}{l} 0\\ X^k_{t} \end{array} \right] \in {\mathfrak {R}}^{(N+1)J\times N}$ with

$$\begin{aligned} X^k_{t}=\left[ \begin{array}{llll} \mathbf{x}_{1t}^1- \mathbf{x}_{1t}^k &{}0 &{} &{} 0 \\ 0 &{} \ddots &{} \ddots &{} \vdots \\ &{} \ddots &{} \ddots &{} 0\\ 0 &{}\cdots &{} 0&{} \mathbf{x}_{Nt}^1- \mathbf{x}_{Nt}^k \end{array} \right] \in {\mathfrak {R}}^{NJ\times N}. \end{aligned}$$

Then, the constraints of the problem (6) can be expressed as follows

$$\begin{aligned} {\varvec{\xi }}_t^k\ge 0,\quad {\mathbf{X}_t^k}^{\top }\tilde{\mathbf{w}}\ge \mathbf{e}-{\varvec{\xi }}_t^k, \ t=1,\ldots ,T,\ k=2,\ldots ,K. \end{aligned}$$

With this notation, the Lagrangian function associated to formulation (6) is given by

$$\begin{aligned} L(\tilde{\mathbf{w}},{\varvec{\xi }}_t^k,{\varvec{\alpha }}_t^k,\mathbf{s}_t^k)= & {} \frac{1}{2}\tilde{\mathbf{w}}^\top {\mathcal{Q}}(\theta ) \tilde{\mathbf{w}}+\sum _{t=1}^T\sum _{k=2}^K\left[ C({\varvec{\xi }}_{t}^k)^\top \mathbf{e}-{{\varvec{\alpha }}_t^k}^\top ({\mathbf{X}_t^k}^{\top }\tilde{\mathbf{w}}- \mathbf{e}+{\varvec{\xi }}_t^k)\right. \nonumber \\&\left. -({\varvec{\xi }}_{t}^k)^\top \mathbf{s}_t^k\right] . \end{aligned}$$

(14)

Then, Problem (6) can be written equivalently as

$$\begin{aligned} \min _{\tilde{\mathbf{w}},{\varvec{\xi }}_t^k}\max _{{\varvec{\alpha }}_t^k,\mathbf{s}_t^k}\{L(\tilde{\mathbf{w}},{\varvec{\xi }}_t^k,{\varvec{\alpha }}_t^k,\mathbf{s}_t^k):{\varvec{\alpha }}_t^k,\mathbf{s}_t^k\ge 0,\, t=1,\ldots ,T,\, k=2,\ldots ,K \}. \end{aligned}$$

Hence, the dual formulation (see eg, Bertsekas, 1982) of (6) is given by

$$\begin{aligned} \max _{{\varvec{\alpha }}_t^k,\mathbf{s}_t^k}\min _{\tilde{\mathbf{w}},{\varvec{\xi }}_t^k}\{L(\tilde{\mathbf{w}},{\varvec{\xi }}_t^k,{\varvec{\alpha }}_t^k,\mathbf{s}_t^k):{\varvec{\alpha }}_t^k,\mathbf{s}_t^k\ge 0,\,t=1,\ldots ,T,\, k=2,\ldots ,K \}. \end{aligned}$$

The above expression enables us to compute the dual problem based only on the Lagrange multipliers $\varvec{\alpha }$. The first-order conditions of the inner minimization problem yields to

$$\begin{aligned} \nabla _{\tilde{\mathbf{w}}}L(\tilde{\mathbf{w}},{\varvec{\xi }}_t^k,{\varvec{\alpha }}_t^k,\mathbf{s}_t^k)= & {} {\mathcal{Q}}(\theta ) \tilde{\mathbf{w}}-\sum _{t=1}^T\sum _{k=2}^K\mathbf{X}_t^k {\varvec{\alpha }}_t^k=0,\end{aligned}$$

(15)

$$\begin{aligned} \nabla _{{\varvec{\xi }}_t^k}L(\tilde{\mathbf{w}},{\varvec{\xi }}_t^k,{\varvec{\alpha }}_t^k,\mathbf{s}_t^k)= & {} C\mathbf{e}-{\varvec{\alpha }}_t^k-\mathbf{s}_t^k=0. \end{aligned}$$

(16)

Since $\mathbf{s}_t^k\ge 0$, from (16) it follows that ${\varvec{\alpha }}_t^k\le C\mathbf{e}$ for $t=1,\ldots ,T$, and $k=2,\ldots ,K$.

Remark 1

Note that using (1), (15), and the notation of $\mathbf{X}_t^k$, we have that

$$\begin{aligned} \mathbf{w}_0=\frac{1}{N}\sum _{i=1}^N \mathbf{w}_i. \end{aligned}$$

On the other hand, by using (15) and (16) in (14), we obtain that

$$\begin{aligned} L(\tilde{\mathbf{w}},{\varvec{\xi }}_t^k,{\varvec{\alpha }}_t^k,\mathbf{s}_t^k)= \sum _{t=1}^T\sum _{k=2}^K{{\varvec{\alpha }}_t^k}^\top \mathbf{e}-\frac{1}{2}\left\| {\mathcal{Q}}(\theta )^{1/2} \tilde{\mathbf{w}}\right\| ^2. \end{aligned}$$

Since ${\mathcal{Q}}(\theta )$ is nonsingular, it follows from (15) that the above expression can be written as

$$\begin{aligned} L(\tilde{\mathbf{w}},{\varvec{\xi }}_t^k,{\varvec{\alpha }}_t^k,\mathbf{s}_t^k)= \sum _{t=1}^T\sum _{k=2}^K{{\varvec{\alpha }}_t^k}^\top \mathbf{e}-\frac{1}{2}\left\| {\mathcal{Q}}(\theta )^{-1/2} \sum _{t=1}^T\sum _{k=2}^K\mathbf{X}_t^k {\varvec{\alpha }}_t^k\right\| ^2. \end{aligned}$$

(17)

The following result allows us to rewrite the above equality.

Proposition 2

For $\theta >0$, let $\widetilde{{\mathcal{Q}}}(\theta )= NI_{JN}+\theta \mathcal{J}\in {\mathfrak {R}}^{JN\times JN}$, where $I_{JN}$ denotes the identity matrix of size JN and

$$\begin{aligned} \mathcal{J}= \left[ \begin{array}{lll} I_J &{} \cdots &{} I_J \\ \vdots &{}\ddots &{}\vdots \\ I_J &{} \cdots &{} I_J \end{array} \right] \in {\mathfrak {R}}^{JN\times JN}. \end{aligned}$$

Then, there exists a symmetric matrix $\widetilde{{\mathcal{Q}}}(\theta )^{1/2}$ satisfying $(\widetilde{{\mathcal{Q}}}(\theta )^{1/2})^2=\widetilde{{\mathcal{Q}}}(\theta )$.

Proof

Let

$$\begin{aligned} \widetilde{{\mathcal{Q}}}(\theta )^{1/2}= \sqrt{N}I_{JN}-\frac{1-\sqrt{1+\theta }}{\sqrt{N}}\mathcal{J}. \end{aligned}$$

(18)

Note that $\mathcal{J}^2=NJ$. Then,

$$\begin{aligned} (\widetilde{{\mathcal{Q}}}(\theta )^{1/2})^2=NI_{JN}-2(1-\sqrt{1+\theta })\mathcal{J}+\frac{(1-\sqrt{1+\theta })^2}{N}(N\mathcal{J})=NI_{JN}+\theta \mathcal{J}. \end{aligned}$$

$\square$

By using the relation (13), Proposition 2, and the definition of $\mathbf{X}_t^k$, the expression (17) reduces to

$$\begin{aligned} L(\tilde{\mathbf{w}},{\varvec{\xi }}_t^k,{\varvec{\alpha }}_t^k,\mathbf{s}_t^k)= \sum _{t=1}^T\sum _{k=2}^K{{\varvec{\alpha }}_t^k}^\top \mathbf{e}-\frac{1}{2N(\theta +1)}\left\| \widetilde{{\mathcal{Q}}}(\theta )^{1/2} \sum _{t=1}^T\sum _{k=2}^K X_t^k {\varvec{\alpha }}_t^k\right\| ^2. \end{aligned}$$

Hence, the dual formulation is given by

$$\begin{aligned} \begin{array}{ll} \max _{{\varvec{\alpha }}_t^k\in {\mathfrak {R}}^N}&{}\ \sum\limits_{t=1}^{T}\sum\limits_{k=2}^{K}{{\varvec{\alpha }}_t^k}^\top \mathbf{e}-\frac{1}{2N(\theta +1)}\left\| \widetilde{{\mathcal{Q}}}(\theta )^{1/2} \sum _{t=1}^T\sum _{k=2}^K X_t^k {\varvec{\alpha }}_t^k\right\| ^2\\ \text{ s.t. }&{}\quad 0\le {\varvec{\alpha }}_t^k\le C\mathbf{e},\ t=1,\ldots ,T,\ k=2,\ldots ,K. \end{array} \end{aligned}$$

(19)

Remark 2

From (15) and (13), it follows that

$$\begin{aligned} \mathbf{w}_0=\frac{1}{N}\sum _{i=1}^N\sum _{t=1}^T\sum _{k=2}^K (\mathbf{x}_{it}^1-\mathbf{x}_{it}^k) { \alpha }_{it}^k, \end{aligned}$$

and

$$\begin{aligned} \mathbf{w}_i=\frac{1}{N(\theta +1)}\left( (\theta +N)\sum _{t=1}^T\sum _{k=2}^K (\mathbf{x}_{it}^1-\mathbf{x}_{it}^k) { \alpha }_{it}^k+\theta \sum _{j=1,j\ne i}^N\sum _{t=1}^T\sum _{k=2}^K (\mathbf{x}_{jt}^1-\mathbf{x}_{jt}^k) { \alpha }_{jt}^k\right) , \end{aligned}$$

(20)

for $i=1,\ldots ,N$.