Let A be a d by n matrix, d < n. Let T = T^n-1 be the standard regular simplex in R^n. We count the faces of the projected simplex AT in the case where the projection is random, the dimension d is large and n and d are comparable: d ~ dn, d in (0, 1). The projector A is chosen uniformly at random from the Grassmann manifold of d-dimensional orthoprojectors of R^n. We derive ?N( d) > 0 with the property that, for any ? < ? N( deter), with overwhelming probability for large d, the number of k-dimensional faces of P = AT is exactly the same as for T, for 0<=k<= ?d. This implies that P is [?d]-neighborly, and its skeleton Skel[? d] ( P) is combinatorially equivalent to Skel[?d] (T). We display graphs of ?N. We also study a weaker notion of neighborliness it asks if the k-faces are all simplicial and if the numbers of k-dimensional faces fk(P) >= fk(T)(1-e). This was already considered by Vershik and Sporyshev, who obtained qualitative results about the existence of a threshold ? VS(d) > 0 at which phase transition occurs in k/d. We compute and display ?VS and compare to ?N. Our results imply that the convex hull of n Gaussian samples in R^d, with n large and proportional to d, ‘looks like a simplex’ in the following sense. In a typical realization of such a high-dimensional Gaussian point cloud d~ dn, all points are on the boundary of the convex hull, and all pairwise line segments, triangles, quadrangles, …, [?d]-angles are on the boundary, for ?<? N(d/n). Our results also quantify a precise phase transition in the ability of linear programming to find the sparsest nonnegative solution to typical systems of underdetermined linear equations; when there is a solution with fewer than ?VS(d/n)d nonzeros, linear programming will find that solution.

Overcomplete representations are attracting interest in signal processing theory, particularly due to their potential to generate sparse representations of signals. However, in general, the problem of finding sparse representations must be unstable in the presence of noise. This paper establishes the possibility of stable recovery under a combination of sufficient sparsity and favorable structure of the overcomplete system. Considering an ideal underlying signal that has a sufficiently sparse representation, it is assumed that only a noisy version of it can be observed. Assuming further that the overcomplete system is incoherent, it is shown that the optimally sparse approximation to the noisy data differs from the optimally sparse decomposition of the ideal noiseless signal by at most a constant multiple of the noise level. As this optimal-sparsity method requires heavy (combinatorial) computational effort, approximation algorithms are considered. It is shown that similar stability is also available using the basis and the matching pursuit algorithms. Furthermore, it is shown that these methods result in sparse approximation of the noisy data that contains only terms also appearing in the unique sparsest representation of the ideal noiseless sparse signal.

Consider an underdetermined system of linear equations y = Ax with known d*n matrix A and known y. We seek the sparsest nonnegative solution, i.e. the nonnegative x with fewest nonzeros satisfying y = Ax. In general this problem is NP-hard. However, for many matrices A there is a threshold phenomenon: if the sparsest solution is sufficiently sparse, it can be found by linear programming. In classical convex polytope theory, a polytope P is called k-neighborly if every set of k vertices of P span a face of P. Let aj denote the j-th column of A, 1<=_j<=_n, let a0 = 0 and let P denote the convex hull of the aj . We say P is outwardly k-neighborly if every subset of k vertices not including 0 spans a face of P. We show that outward k-neighborliness is completely equivalent to the statement that, whenever y = Ax has a nonnegative solution with at most k nonzeros, it is the nonnegative solution to y = Ax having minimal sum. Using this and classical results on polytope neighborliness we obtain two types of corollaries. First, because many [d/2]-neighborly polytopes are known, there are many systems where the sparsest solution is available by convex optimization rather than combinatorial optimization - provided the answer has fewer nonzeros than half the number of equations. We mention examples involving incompletely-observed Fourier transforms and Laplace transforms. Second, results on classical neighborliness of high-dimensional randomly-projected simplices imply that, if A is a typical uniformly-distributed random orthoprojector with n = 2d and n large, the sparsest nonnegative solution to y = Ax can be found by linear programming provided it has fewer nonzeros than 1/8 the number of equations. We also consider a notion of weak neighborliness, in which the overwhelming majority of k-sets of aj's not containing 0 span a face. This implies that most nonnegative vectors x with k nonzeros are uniquely determined by y = Ax. As a corollary of recent work counting faces of random simplices, it is known that most polytopes P generated by large n by 2n uniformly-distributed orthoprojectors A are weakly k-neighborly with k _~.558n. We infer that for most n by 2n underdetermined systems having a sparse solution with fewer nonzeros than roughly half the number of equations, the sparsest solution can be found by linear programming.

Given a signal S ( R^N and a full-rank matrix D ( R^NL with N<L, we define the signal’s over-complete representation as a ( R^L satisfying S=Da. Among the infinitely many solutions of this under-determined linear system of equations, we have special interest in the sparsest representation, i.e., the one minimizing ||a||0. This problem has a combinatorial flavor to it, and its direct solution is impossible even for moderate L. Approximation algorithms are thus required, and one such appealing technique is the basis pursuit (BP) algorithm. This algorithm has been the focus of recent theoretical research effort. It was found that if indeed the representation is sparse enough, BP finds it accurately. When an error is permitted in the composition of the signal, we no longer require exact equality S=Da. The BP has been extended to treat this case, leading to a denoizing algorithm. The natural question to pose is how the abovementioned theoretical results generalize to this more practical mode of operation. In this paper we propose such a generalization. The behavior of the basis pursuit in the presence of noise has been the subject of two independent very wide contributions released for publication very recently. This paper is another contribution in this direction, but as opposed to the others mentioned, this paper aims to present a somewhat simplified picture of the topic, and thus could be referred to as a primer to this field. Specifically, we establish here the stability of the BP in the presence of noise for sparse enough representations. We study both the case of a general dictionary D, and a special case where D is built as a union of orthonormal bases. This work is a direct generalization of noiseless BP study, and indeed, when the noise power is reduced to zero, we obtain the known results of the noiseless BP.

In compressed sensing, one takes n < N samples of an N -dimensional vector x0 using an n × N matrix A, obtaining un-dersampled measurements y = Ax0 . For random matrices with Gaussian i.i.d entries, it is known that, when x0 is k-sparse, there is a precisely determined phase transition: for a certain region in the (k/n, n/N )-phase diagram, convex optimization min ||x||_1 subject to y = Ax, x ∈ X^N typically finds the sparsest solution, while outside that region, it typically fails. It has been shown empirically that the same property – with the same phase transition location – holds for a wide range of non-Gaussian random matrix ensembles. We consider specific deterministic matrices including Spikes and Sines, Spikes and Noiselets, Paley Frames, Delsarte-Goethals Frames, Chirp Sensing Matrices, and Grassmannian Frames. Extensive experiments show that for a typical k-sparse object, convex optimization is successful over a region of the phase diagram that coincides with the region known for Gaussian matrices. In our experiments, we considered coefficients constrained to X^N for four different sets X ∈ {[0, 1], R_+ , R, C}. We establish this finding for each of the associated four phase transitions.