python - Django Rest框架自定义过滤后端数据重复

Question

我正在尝试使我的自定义过滤器和排序后端与 django Rest 框架中的默认搜索后端一起使用。过滤和排序彼此完美配合，但是当搜索包含在查询中并且我尝试按对象名称对查询进行排序时，就会发生数据重复。

我尝试打印查询和查询大小，但当我将其记录在过滤器中时似乎没问题，但在响应中我有不同的对象计数(例如过滤器查询中的 79 个对象，最终结果中的 170 个重复对象)

这是我的过滤器集类

class PhonesFilterSet(rest_filters.FilterSet):
    brands = InListFilter(field_name='brand__id')
    os_ids = InListFilter(field_name='versions__os')
    version_ids = InListFilter(field_name='versions')
    launched_year_gte = rest_filters.NumberFilter(field_name='phone_launched_date__year', lookup_expr='gte')
    ram_gte = rest_filters.NumberFilter(field_name='internal_memories__value', method='get_rams')
    ram_memory_unit = rest_filters.NumberFilter(field_name='internal_memories__units', method='get_ram_units')

    def get_rams(self, queryset, name, value):
        #here is the problem filter
        #that not works with ordering by name
        q=queryset.filter(Q(internal_memories__memory_type=1) & Q(internal_memories__value__gte=value))
        print('filter_set', len(q))
        print('filter_set_query', q.query)
        return q


    def get_ram_units(self, queryset, name, value):
        return queryset.filter(Q(internal_memories__memory_type=1) & Q(internal_memories__units=value))


    class Meta:
        model = Phone
        fields = ['brands', 'os_ids', 'version_ids', 'status', 'ram_gte']

我的订购类:

class CustomFilterBackend(filters.OrderingFilter):
    allowed_custom_filters = ['ram', 'camera', 'year']

    def get_ordering(self, request, queryset, view):
        params = request.query_params.get(self.ordering_param)

        if params:
            fields = [param.strip() for param in params.split(',')]
            ordering = [f for f in fields if f in self.allowed_custom_filters]
            if ordering:
                return ordering

        # No ordering was included, or all the ordering fields were invalid

        return self.get_default_ordering(view)


    def filter_queryset(self, request, queryset, view):
        ordering = self.get_ordering(request, queryset, view)
        if ordering:
            if 'ram' in ordering:
                max_ram = Max('internal_memories__value', filter=Q(internal_memories__memory_type=1))
                queryset = queryset.annotate(max_ram=max_ram).order_by('-max_ram')
            elif 'camera' in ordering:
                max_camera = Max('camera_pixels__megapixels', filter=Q(camera_pixels__camera_type=0))    
                queryset = queryset.annotate(max_camera=max_camera).order_by('-max_camera')            
            elif 'year' in ordering:
                queryset = queryset.filter(~Q(phone_released_date=None)).order_by('-phone_released_date__year')
            elif 'name' in ordering:
                #here is the problem ordering
                #thats not working with filter
                #with one to many relations
                queryset = queryset.order_by('-brand__name', '-model__name')


        return queryset

View 集类:

class PhoneViewSet(viewsets.ModelViewSet):
    queryset = Phone.objects.all()
    serializer_class = PhoneSerializer
    filter_backends = (filters.SearchFilter, CustomFilterBackend, django_filters.rest_framework.DjangoFilterBackend)
    search_fields = ('brand__name', 'model__name')
    ordering_fields = ('brand__name', 'model__name')
    filter_class = PhonesFilterSet

因此，当我使用过滤器和搜索应用排序时，我希望不会出现数据重复。我的问题是为什么过滤器和响应中的对象数量不同，其中数据变得重复？我不知道从哪里开始调试。提前致谢。

Answer 1

使用 distinct() 应该可以解决这个问题:

Returns a new QuerySet that uses SELECT DISTINCT in its SQL query. This eliminates duplicate rows from the query results.

By default, a QuerySet will not eliminate duplicate rows. In practice, this is rarely a problem, because simple queries such as Blog.objects.all() don’t introduce the possibility of duplicate result rows. However, if your query spans multiple tables, it’s possible to get duplicate results when a QuerySet is evaluated. That’s when you’d use distinct().

但请注意，您仍然可能会得到重复的结果:

Any fields used in an order_by() call are included in the SQL SELECT columns. This can sometimes lead to unexpected results when used in conjunction with distinct(). If you order by fields from a related model, those fields will be added to the selected columns and they may make otherwise duplicate rows appear to be distinct. Since the extra columns don’t appear in the returned results (they are only there to support ordering), it sometimes looks like non-distinct results are being returned.

https://docs.djangoproject.com/en/2.2/ref/models/querysets/#django.db.models.query.QuerySet.distinct

如果您使用的是 PostgreSQL，则可以指定应应用 DISTINCT 的字段名称。这可能会有所帮助。 (我不确定。)有关此内容的更多信息，请参阅上面的链接。

所以，我会在您评论说遇到问题的方法中返回queryset.distinct()。我不会总是应用它(正如我在上面的调试评论中所写的那样)，因为简单查询不需要它。