CitySacpes数据集转COCO格式（labelIds.png转json）

导语：

CSDN上有位大神写了Cityscapes转COCO格式的代码，但是代码是将xxx_gtFine_instanceIds.png即实例分割标注转换成COCO格式。然而很多我需要的类别，例如道路，人行道，树以及天空等类别是没有的，要得到这些类别需要对语义分割标注即xxx_gtFine_labelIds.png文件进行转换。

所以本文主要针对把CityScapes数据集的语义分割标注xxx_gtFine_labelIds.png转成COCO数据集的json格式。

1、数据格式：

首先简单介绍一下Cityscapes数据集，至于详细说明及数据集百度云下载链接请移步此博客
leftImg8bit是原图文件夹，gtFine是相对应的标注文件。

对于每一张街景图片在gtFine文件下都有对应的如下几张标注文件。其中gtFine_color.png是彩色的可视化结果，gtFine_instancelds.png是实例分割的标注，gtFine_labellds.png是语义分割的标注，对于gtFine_polygons.json是由labelme标注转换得到的json文件，一开始想直接对这个json文件进行读取转换，但是发现test里的标注不完全，所以就放弃了。具体图片如下所展示：

原街景图像：zurich_000006_000019_leftImg8bit.png

大小为2048 * 1024 * 3。

可视化结果：zurich_000006_000019_gtFine_color.png

实例分割：zurich_000006_000019_gtFine_instanceIds.png

语义分割：zurich_000006_000019_gtFine_labelIds.png

肉眼可以明显看到gtFine_instanceIds.png的效果，但是对于gtFine_labelIds.png却有点看不清。这是因为**_gtFine_labelIds.png的每个像素值是0-33，**不同的值代表不同的类，而gtFine_instanceIds.png的每个像素值是 类别*1000。

2、CityScapes转换COCO格式步骤：

这条博客讲得十分清楚、直观且易懂，基本上是手把手教学了，我就不再详细介绍了。按照博主的步骤只需要改动部分代码就可以了。所以我主要说一下使用语义分割标注需要改动的代码部分。
首先是刚刚说到的对于instanceId和labelId的标注，像素点的取值不一样，所以一共有两处需要改动的地方：

（1）首先是将打开的图片改成以语义分割标注"_gtFine_labelIds.png"为后缀的。

def image_trans():
    img_subfolders = os.listdir(IMAGE_DIR)
    image_count = 0
    for sub in img_subfolders:
        # sub_path = sub + '/' + sub
        image_sub_path = os.path.join(IMAGE_DIR, sub)
        for image in os.listdir(image_sub_path):
            img_path = os.path.join(image_sub_path, image)
            ann_name = image.split('_')[0] + '_' + image.split('_')[1] + '_' + image.split('_')[
                2] + '_gtFine_labelIds.png'
            ann_sub_path = os.path.join(ANNOTATION_DIR, sub)
            ann_path = os.path.join(ann_sub_path, ann_name)
            if os.path.exists(ann_path):
                pic = cv2.imread(img_path)
                h, w = pic.shape[:2]
                new_w = w * pic_scale
                new_h = new_w / 2
                top = int((h_bias * h - new_h) / 2)
                bottom = int((h_bias * h + new_h) / 2)
                left = int((w - new_w) / 2)
                right = int((w + new_w) / 2)
                roi = pic[top:bottom, left:right]
                img_save_path = os.path.join(IMAGE_SAVE_DIR, image)
                cv2.imwrite(img_save_path, roi)
                annotation = cv2.imread(ann_path, -1)
                ann_roi = annotation[top:bottom, left:right]
                ann_save_path = os.path.join(ANNOTATION_SAVE_DIR, ann_name)
                cv2.imwrite(ann_save_path, ann_roi)
            else:
                print(image + '  do not have instance annotation')
            print(image_count)
            image_count += 1

（2）其次是将class_id的的获得方式改为不地板除1000，因为对于_labelId.png来说，像素值的范围在0-33之间，无需这个步骤。
对于实例分割的mask，尝试输出unique之后的值如下

munster_000158_000019_gtFine_instanceIds.png
[    1     3     4     5     7     8    11    17    19    20    21    22
    23    33 25000 26000 26001 26002 26003 26004 26005 26006 26007 26008
 26009 26010 26011 26012 26013 26014 26015 26016 32000 33000 33001]

25000代表该块属于id=25类别的第0个物体

25000代表该块属于id=25类别的第0个物体

26001代表该块属于id=26类别的第1个物体

代码如下：

def masks_generator(imges, ann_path):
    global idx
    pic_count = 0
    for pic_name in imges:
        image_name = pic_name.split('.')[0]
        ann_folder = os.path.join(INSTANCE_DIR, image_name)
        if not os.path.exists(ann_folder):
            os.mkdir(ann_folder)
        print("pic_name:", pic_name)
        annotation_name = pic_name.split('_')[0] + '_' + pic_name.split('_')[1] + '_' + pic_name.split('_')[
            2] + '_gtFine_labelIds.png'
        # annotation_name = image_name + '_instanceIds.png'
        print(annotation_name)
        annotation = cv2.imread(os.path.join(ann_path, annotation_name), -1)
        h, w = annotation.shape[:2]
        ids = np.unique(annotation)
        print(ids)
        for id in ids:
            if id in background_label:
                continue
            else:
                #class_id = id // 1000
                class_id = id
                #改成你需要的类别，其余对应的地方也需要改
                if class_id == 26:
                    instance_class = 'car'
                elif class_id == 24:
                    instance_class = 'pedestrian'
                elif class_id == 27:
                    instance_class = 'truck'
                elif class_id == 28:
                    instance_class = 'bus'
                elif class_id == 25:
                    instance_class = 'rider'
                elif class_id == 7:
                    instance_class = 'road'
                elif class_id == 8:
                    instance_class = 'sidewalk'
                elif class_id == 11:
                    instance_class = 'building'
                elif class_id == 12:
                    instance_class = 'wall'
                elif class_id == 13:
                    instance_class = 'fence'
                elif class_id == 17:
                    instance_class = 'pole'
                elif class_id == 19:
                    instance_class = 'traffic light'
                elif class_id == 20:
                    instance_class = 'traffic sign'
                elif class_id == 21:
                    instance_class = 'vegetation '
                elif class_id == 22:
                    instance_class = 'terrain '
                elif class_id == 23:
                    instance_class = 'sky'
                elif class_id == 31:
                    instance_class = 'train'
                elif class_id == 32:
                    instance_class = 'motorcycle'
                elif class_id == 33:
                    instance_class = 'bicycle'
                else:
                    continue
            instance_mask = np.zeros((h, w, 3), dtype=np.uint8)
            mask = annotation == id
            instance_mask[mask] = 255
            mask_name = image_name + '_' + instance_class + '_' + str(idx) + '.png'
            cv2.imwrite(os.path.join(ann_folder, mask_name), instance_mask)
            idx += 1
        pic_count += 1
        print(pic_count)

（3）完整代码。以上是主要需要修改的两处代码，其余的代码按照你所需要的标签来转换，这里给出转换了19个类别的完整代码

import sys

if "/opt/ros/kinetic/lib/python2.7/dist-packages" in sys.path:
    sys.path.remove("/opt/ros/kinetic/lib/python2.7/dist-packages")

import cv2
import numpy as np
import os, glob
from shutil import copyfile
import datetime
import json
import os
import re
import fnmatch
from PIL import Image
import numpy as np
from pycococreatortools import pycococreatortools


ROOT_DIR = '/home/ubt/devdata/city_coco/data/city'
IMAGE_DIR = os.path.join(ROOT_DIR, "leftImg8bit/val")
ANNOTATION_DIR = os.path.join(ROOT_DIR, "gtFine/val")
ANNOTATION_SAVE_DIR = os.path.join(ROOT_DIR, "annotations")
INSTANCE_DIR = os.path.join(ROOT_DIR, "gtFine/val")
IMAGE_SAVE_DIR = os.path.join(ROOT_DIR, "val_images")

INFO = {
    "description": "Cityscapes_Instance Dataset",
    "url": "https://github.com/waspinator/pycococreator",
    "version": "0.1.0",
    "year": "2020",
    "contributor": "mallika",
    "date_created": "2020-12-26 19:19:19.123456"
}

LICENSES = [
    {
        "id": 1,
        "name": "Attribution-NonCommercial-ShareAlike License",
        "url": "http://creativecommons.org/licenses/by-nc-sa/2.0/"
    }
]

CATEGORIES = [
    {
        'id': 1,
        'name': 'car',
        'supercategory': 'cityscapes',
    },
    {
        'id': 2,
        'name': 'pedestrian',
        'supercategory': 'cityscapes',
    },
    {
        'id': 3,
        'name': 'truck',
        'supercategory': 'cityscapes',
    },
    {
        'id': 4,
        'name': 'bus',
        'supercategory': 'cityscapes',
    },
    {
        'id': 5,
        'name': 'rider',
        'supercategory': 'cityscapes',
    },
    {
        'id': 6,
        'name': 'road',
        'supercategory': 'cityscapes',
    },
    {
        'id': 7,
        'name': 'sidewalk',
        'supercategory': 'cityscapes',
    },
    {
        'id': 8,
        'name': 'building',
        'supercategory': 'cityscapes',
    },
    {
        'id': 9,
        'name': 'wall',
        'supercategory': 'cityscapes',
    },
    {
        'id': 10,
        'name': 'fence',
        'supercategory': 'cityscapes',
    },
    {
        'id': 11,
        'name': 'pole',
        'supercategory': 'cityscapes',
    },
    {
        'id': 12,
        'name': 'traffic light',
        'supercategory': 'cityscapes',
    },
    {
        'id': 13,
        'name': 'traffic sign',
        'supercategory': 'cityscapes',
    },
    {
        'id': 14,
        'name': 'vegetation',
        'supercategory': 'cityscapes',
    },
    {
        'id': 15,
        'name': 'terrain',
        'supercategory': 'cityscapes',
    },
    {
        'id': 16,
        'name': 'sky',
        'supercategory': 'cityscapes',
    },
    {
        'id': 17,
        'name': 'train',
        'supercategory': 'cityscapes',
    },
    {
        'id': 18,
        'name': 'motorcycle',
        'supercategory': 'cityscapes',
    },
    {
        'id': 19,
        'name': 'bicycle',
        'supercategory': 'cityscapes',
    }
]


background_label = [-1, 0, 1, 2, 3, 4, 5, 6, 9, 10, 14, 15, 16, 18, 29, 30]
idx = 0
pic_scale = 1.0
h_bias = 1.0


def image_trans():
    img_subfolders = os.listdir(IMAGE_DIR)
    image_count = 0
    for sub in img_subfolders:
        # sub_path = sub + '/' + sub
        image_sub_path = os.path.join(IMAGE_DIR, sub)
        for image in os.listdir(image_sub_path):
            img_path = os.path.join(image_sub_path, image)
            ann_name = image.split('_')[0] + '_' + image.split('_')[1] + '_' + image.split('_')[
                2] + '_gtFine_labelIds.png'
            ann_sub_path = os.path.join(ANNOTATION_DIR, sub)
            ann_path = os.path.join(ann_sub_path, ann_name)
            if os.path.exists(ann_path):
                pic = cv2.imread(img_path)
                h, w = pic.shape[:2]
                new_w = w * pic_scale
                new_h = new_w / 2
                top = int((h_bias * h - new_h) / 2)
                bottom = int((h_bias * h + new_h) / 2)
                left = int((w - new_w) / 2)
                right = int((w + new_w) / 2)
                roi = pic[top:bottom, left:right]
                img_save_path = os.path.join(IMAGE_SAVE_DIR, image)
                cv2.imwrite(img_save_path, roi)
                annotation = cv2.imread(ann_path, -1)
                ann_roi = annotation[top:bottom, left:right]
                ann_save_path = os.path.join(ANNOTATION_SAVE_DIR, ann_name)
                cv2.imwrite(ann_save_path, ann_roi)
            else:
                print(image + '  do not have instance annotation')
            print(image_count)
            image_count += 1


def data_loader():
    imgs = os.listdir(IMAGE_SAVE_DIR)
    masks_generator(imgs, ANNOTATION_SAVE_DIR)


def masks_generator(imges, ann_path):
    global idx
    pic_count = 0
    for pic_name in imges:
        image_name = pic_name.split('.')[0]
        ann_folder = os.path.join(INSTANCE_DIR, image_name)
        if not os.path.exists(ann_folder):
            os.mkdir(ann_folder)
        print("pic_name:", pic_name)
        annotation_name = pic_name.split('_')[0] + '_' + pic_name.split('_')[1] + '_' + pic_name.split('_')[
            2] + '_gtFine_labelIds.png'
        # annotation_name = image_name + '_instanceIds.png'
        print(annotation_name)
        annotation = cv2.imread(os.path.join(ann_path, annotation_name), -1)
        h, w = annotation.shape[:2]
        ids = np.unique(annotation)
        print(ids)
        for id in ids:
            if id in background_label:
                continue
            else:
                #class_id = id // 1000
                class_id = id
                if class_id == 26:
                    instance_class = 'car'
                elif class_id == 24:
                    instance_class = 'pedestrian'
                elif class_id == 27:
                    instance_class = 'truck'
                elif class_id == 28:
                    instance_class = 'bus'
                elif class_id == 25:
                    instance_class = 'rider'
                elif class_id == 7:
                    instance_class = 'road'
                elif class_id == 8:
                    instance_class = 'sidewalk'
                elif class_id == 11:
                    instance_class = 'building'
                elif class_id == 12:
                    instance_class = 'wall'
                elif class_id == 13:
                    instance_class = 'fence'
                elif class_id == 17:
                    instance_class = 'pole'
                elif class_id == 19:
                    instance_class = 'traffic light'
                elif class_id == 20:
                    instance_class = 'traffic sign'
                elif class_id == 21:
                    instance_class = 'vegetation '
                elif class_id == 22:
                    instance_class = 'terrain '
                elif class_id == 23:
                    instance_class = 'sky'
                elif class_id == 31:
                    instance_class = 'train'
                elif class_id == 32:
                    instance_class = 'motorcycle'
                elif class_id == 33:
                    instance_class = 'bicycle'
                else:
                    continue
            instance_mask = np.zeros((h, w, 3), dtype=np.uint8)
            mask = annotation == id
            instance_mask[mask] = 255
            mask_name = image_name + '_' + instance_class + '_' + str(idx) + '.png'
            cv2.imwrite(os.path.join(ann_folder, mask_name), instance_mask)
            idx += 1
        pic_count += 1
        print(pic_count)


def json_generate():
    car = 0
    pedestrian = 0
    truck = 0
    bus = 0
    rider = 0
    road = 0
    sidewalk = 0
    building = 0
    wall = 0
    fence = 0
    pole = 0
    traffic_light = 0
    traffic_sign = 0
    vegetation = 0
    terrain = 0
    sky = 0
    train = 0
    motorcycle = 0
    bicycle = 0
    files = os.listdir(IMAGE_SAVE_DIR)

    coco_output = {
        "info": INFO,
        "licenses": LICENSES,
        "categories": CATEGORIES,
        "images": [],
        "annotations": []
    }

    image_id = 1
    segmentation_id = 1

    # go through each image
    for image_filename in files:
        image_name = image_filename.split('.')[0]
        image_path = os.path.join(IMAGE_SAVE_DIR, image_filename)
        image = Image.open(image_path)
        image_info = pycococreatortools.create_image_info(
            image_id, os.path.basename(image_filename), image.size)
        coco_output["images"].append(image_info)
        print(image_filename)
        annotation_sub_path = os.path.join(INSTANCE_DIR, image_name)
        ann_files = os.listdir(annotation_sub_path)
        if len(ann_files) == 0:
            print("no avaliable annotation")
            continue
        else:
            for annotation_filename in ann_files:
                annotation_path = os.path.join(annotation_sub_path, annotation_filename)
                for x in CATEGORIES:
                    if x['name'] in annotation_filename:
                        class_id = x['id']
                        break
                # class_id = [x['id'] for x in CATEGORIES if x['name'] in annotation_filename][0]
                if class_id == 1:
                    car += 1
                elif class_id == 2:
                    pedestrian += 1
                elif class_id == 3:
                    truck += 1
                elif class_id == 4:
                    bus += 1
                elif class_id == 5:
                    rider += 1
                elif class_id == 6:
                    road += 1
                elif class_id == 7:
                    sidewalk += 1
                elif class_id == 8:
                    building += 1
                elif class_id == 9:
                    wall += 1
                elif class_id == 10:
                    fence += 1
                elif class_id == 11:
                    pole += 1
                elif class_id == 12:
                    traffic_light += 1
                elif class_id == 13:
                    traffic_sign += 1
                elif class_id == 14:
                    vegetation += 1
                elif class_id == 15:
                    terrain += 1
                elif class_id == 16:
                    sky += 1
                elif class_id == 17:
                    train += 1
                elif class_id == 18:
                    motorcycle += 1
                elif class_id == 19:
                    bicycle += 1
                else:
                    print('illegal class id')
                category_info = {'id': class_id, 'is_crowd': 'crowd' in image_filename}
                binary_mask = np.asarray(Image.open(annotation_path)
                                         .convert('1')).astype(np.uint8)

                annotation_info = pycococreatortools.create_annotation_info(
                    segmentation_id, image_id, category_info, binary_mask,
                    image.size, tolerance=2)

                if annotation_info is not None:
                    coco_output["annotations"].append(annotation_info)

                    segmentation_id = segmentation_id + 1

            image_id = image_id + 1
            print(image_id)

    with open('{}/val_modified.json'.format(ROOT_DIR), 'w') as output_json_file:
        json.dump(coco_output, output_json_file)
    print(car, pedestrian, truck, bus, rider, road, sidewalk, building, wall, fence, pole, traffic_light, traffic_sign,
          vegetation, terrain, sky, train, motorcycle, bicycle)


if __name__ == "__main__":
    image_trans()
    data_loader()
    json_generate()

训练集（train+test）4500张图片转换完成后的各类别个数:
验证集（val）500张图片转换完成后的各类别个数：
然后大家就可以拿着生成的json文件愉快的训练了，至于训练集和验证集的图片直接使用train_images和val_images里面的就可以啦！

本文地址：https://blog.csdn.net/weixin_41070993/article/details/111992338