Automatic 3D extraction of building roofs from remotely sensed data is important for many applications including city modeling. This paper proposes a new method for automatic 3D roof extraction through an effective integration of LIDAR (Light Detection And Ranging) data and multispectral orthoimagery. Using the ground height from a DEM (Digital Elevation Model), the raw LIDAR points are separated into two groups. The first group contains the ground points that are exploited to constitute a `ground mask'. The second group contains the non-ground points which are segmented using an innovative image line guided segmentation technique to extract the roof planes. The image lines extracted from the grey-scale version of the orthoimage are classified into several classes such as `ground', `tree', `roof edge' and `roof ridge' using the ground mask and colour and texture information from the orthoimagery. During roof plane extraction the lines from the later two classes are used to fit roof planes to the neighbouring non-ground LIDAR points. Finally, a new rule-based procedure is applied to remove planes constructed on trees. Experimental results show that the proposed method successfully removes vegetation and offers high extraction rates.