The non-backtracking matrix of a graph is a non-symmetric matrix on the oriented edge of a graph which has interesting algebraic properties and appears notably in connection with the Ihara zeta function and in some generalizations of Ramanujan graphs. It has also be proposed recently in the context of community detection. In this talk, we will study the largest eigenvalues of this matrix for the Erdos-Renyi graph G(n,c/n) and other simple inhomogeneous random graphs. This is a joint work with Marc Lelarge and Laurent Maussouli.